SCADA Historian: Data Logging and Trend Analysis That Actually Helps
A SCADA system without a historian only tells you what is happening right now. When a pump fails at 3 AM, the operator screen shows the failure. It does not show the gradual increase in bearing temperature over the preceding six hours that would have predicted the failure. When a regulator asks for chlorine residual logs from last month, the SCADA system cannot provide them because it never stored them. When management wants to optimize energy consumption, there is no baseline data to analyze.
A historian fixes all of this. A historian is a time-series database that stores every data point at full resolution, time-stamped and indexed. When you need to troubleshoot a problem, generate a compliance report, or analyze trends, you query the historian and get the complete picture—not just what is on the screen right now.
This guide explains what a historian actually does, why it is different from real-time SCADA data, when you need one, and the build-vs-buy decision between DIY solutions (InfluxDB + Grafana) and integrated SCADA systems with built-in historians.
Real-Time SCADA Data vs Historian Data
The distinction between real-time data and historian data is the single most important concept to understand:
Real-Time Data: The Present Only
Real-time SCADA data is what you see on operator screens and dashboards. It is the current state of equipment. When the SCADA system polls a Modbus device and reads a temperature of 72.5 degrees, that value is displayed on the screen. When the next poll occurs 5 seconds later and reads 72.7 degrees, the screen updates. The previous value (72.5) is overwritten. It is gone forever.
Real-time data is sufficient for monitoring. Operators see current conditions and respond to alarms. But it is insufficient for analysis. You cannot look back in time. You cannot see trends. You cannot troubleshoot past events.
Historian Data: The Complete Record
A historian stores every poll. The 72.5 reading from 10:00:05 is stored. The 72.7 reading from 10:00:10 is stored. Every reading, every time, with a timestamp. This builds a time-series record that can be queried later. "What was the temperature between 2 PM and 4 PM last Tuesday?" The historian returns the complete dataset with timestamps.
The critical difference: real-time data is overwritten. Historian data is never overwritten. Real-time data tells you what is happening now. Historian data tells you what happened over hours, days, months, or years.
The Dashboard Illusion
Some SCADA systems show trend charts on dashboards. These charts appear to show historical data. In many cases, the chart only shows data since the dashboard was opened. If you close the browser and come back tomorrow, the chart is empty. That is not a historian—that is a temporary buffer. A true historian persists data permanently. Even if the server restarts, the data is still there.
What a Historian Actually Does
A historian is not just a database. It is a specialized time-series database with features optimized for process data:
Time-Series Storage
Historians store data as time-ordered sequences. Each data point has a timestamp, a value, and quality flags (good/bad/uncertain). This structure enables efficient time-range queries: "Give me all values between time T1 and time T2." General-purpose databases can store time-series data, but they are not optimized for it. Historians use specialized compression algorithms that achieve 10:1 to 20:1 compression ratios while maintaining full precision.
Data Rollup and Downsampling
Raw data at 1-second intervals is valuable for detailed troubleshooting, but querying 10 years of raw data is slow. Historians automatically roll up data: raw data is kept for a period (e.g., 6 months), then aggregated into hourly averages, daily averages, or monthly averages for long-term storage. Queries automatically use the appropriate resolution—raw data for recent time ranges, rolled-up data for long time ranges.
Interpolation and Gap Filling
Communications failures happen. When data is missing, historians can interpolate (estimate values between known points) or fill gaps with the last known value. This produces cleaner trend charts and more accurate analytics.
Annotations and Events
Historians allow you to annotate the timeline. "Pump replaced at this timestamp." "Process configuration changed at this timestamp." Queries can include or exclude data from specific periods. This is critical for accurate analysis—you do not want pump vibration data from before a bearing replacement to skew your post-replacement analysis.
Query Performance
Historians are optimized for time-range queries. Fetching a year of minute-by-minute data from thousands of tags takes seconds, not minutes. This performance enables interactive analytics and dashboards that load historical data on demand.
Why You Need a Historian: Three Use Cases
A historian is not optional for serious SCADA applications. Here is why:
Troubleshooting: See What Led to Failure
When equipment fails, the first question is "what happened before?" A historian answers this. If a pump overheats and trips, the historian shows the temperature trend over the preceding hours. You might see a gradual increase from normal (70°C) to trip temperature (95°C) over 8 hours. This indicates bearing wear, not a sudden blockage. The maintenance response differs: schedule bearing replacement rather than clear a debris blockage.
Without a historian, you only see the trip. You do not see the precursor trend. You are flying blind.
Compliance: Audit Trail Everything
Regulated industries (water treatment, pharmaceuticals, food processing) must prove that processes operated within specified parameters. The environmental agency does not accept "the operator saw it was fine on the screen." They demand logged data with timestamps.
Historian data provides this audit trail. Water systems must log chlorine residual, turbidity, pH, and flow rates continuously. Food processing must log cooking temperatures and times. Pharmaceutical manufacturing must log batch conditions. Without a historian, you cannot generate compliance reports.
Optimization: Data-Driven Decisions
Historical data enables optimization. Energy consumption trends identify inefficient equipment. Pump run time trends predict maintenance needs. Flow patterns help size equipment. Without historical baselines, optimization is guesswork. With historical data, it is analysis.
Example: A lift station runs two pumps alternately. Historian data shows Pump A runs 15 hours per day, Pump B runs 9 hours per day. Both pumps should run roughly 12 hours. The imbalance indicates a problem—perhaps Pump B's check valve is leaking, causing it to run less. Without the historical run-time data, the imbalance is invisible.
Build vs Buy: InfluxDB + Grafana vs Integrated Historian
The build-vs-buy decision is the most common question about historians. The DIY approach uses InfluxDB (or Prometheus) for storage and Grafana for visualization. The buy approach uses a SCADA system with a built-in historian.
DIY Historian: InfluxDB + Grafana
The DIY stack is compelling because the components are free:
- InfluxDB is a time-series database with excellent query performance and built-in retention policies. It handles data rollup and downsampling automatically.
- Grafana is a visualization platform that connects to InfluxDB and produces beautiful dashboards and trend charts.
The total cost is $0 for software. You only pay for hardware. For hobby projects, lab monitoring, and simple applications, this stack works well.
However, the DIY stack has gaps in production environments:
- No SCADA protocol drivers. InfluxDB does not speak Modbus, DNP3, or OPC-UA. You need middleware to poll devices and write to InfluxDB. This middleware is typically a custom Python script or a tool like Telegraf. You are now responsible for maintaining that middleware.
- No alarm management. InfluxDB and Grafana have basic alerting, but they lack the alarm prioritization, acknowledgment, escalation, and notification paths that production SCADA systems require. You will need to build this yourself.
- No redundancy. If your InfluxDB server crashes, you lose data. High-availability InfluxDB requires clustering (Enterprise license) or manual replica management. Commercial SCADA systems include hot-standby redundancy out of the box.
- No audit trail. Compliance requires tamper-evident logging with user tracking. InfluxDB does not provide this out of the box. You must build it.
- Maintenance overhead. You are now responsible for updating InfluxDB, updating Grafana, maintaining the middleware, handling backups, and managing security patches. The $0 software cost is offset by labor cost.
Integrated SCADA Historian
Commercial SCADA systems include historians as part of the platform. Voltrus, Ignition, VTScada, and others have built-in time-series databases that are tightly integrated with the SCADA engine.
The advantages:
- Integrated protocol support. The SCADA system polls devices via Modbus, DNP3, or OPC-UA and writes directly to the internal historian. No middleware required.
- Unified alarm and data. Alarms and historical data are stored in the same system. Correlating an alarm with the data trend is straightforward—query both from the same database.
- Built-in redundancy. Commercial systems support hot-standby server configuration. The historian replicates to the standby server automatically.
- Compliance features. Audit trails, user access logging, and data integrity verification are included.
- Support. When something breaks, you have support to call. With DIY, you are on your own.
The disadvantage: cost. Commercial SCADA systems require licenses. However, when you factor in the labor cost of building and maintaining a DIY solution, the commercial license often pays for itself in the first year.
Historian Retention Strategy
How long should you keep data? The answer balances storage cost, query performance, and analytical value.
Tiered Retention
The standard approach is tiered retention with automatic rollup:
- Raw data (1-5 second interval) for 6-12 months. Used for detailed troubleshooting of recent events.
- 1-minute averaged data for 2-5 years. Used for trend analysis and medium-term optimization.
- Hourly or daily averaged data for 10+ years. Used for long-term compliance reporting and macro-level analysis.
This tiered approach keeps storage manageable while preserving analytical value. The raw data you need for deep troubleshooting is available for the recent past. Long-term trends are preserved at appropriate granularity.
Permanent Retention for Compliance
Some data must be retained permanently. Water treatment disinfectant residual, pharmaceutical batch records, and safety-critical process data often have regulatory retention requirements (sometimes 25+ years). These datasets should be identified in the historian configuration and excluded from automatic rollup or archival deletion.
Archival to Cold Storage
For very long retention (decades), consider archiving to cold storage (Amazon S3 Glacier, Azure Archive Storage). The historian periodically exports data beyond a certain age to compressed archives and deletes from the live database. Archive retrieval takes hours, but the data is preserved for compliance at very low storage cost.
Storage Cost Reality
Storage is cheap. A 2 TB NVMe drive costs less than $150. At 500 MB per day for a medium SCADA system, 2 TB stores 11 years of raw data. The total cost over 11 years is less than $150 in hardware, amortized to $13 per year. The storage cost is negligible compared to the value of the data.
How Voltrus Implements Historian
Voltrus includes a built-in historian that is tightly integrated with the SCADA engine. No separate database server to configure. No middleware to maintain.
Automatic Data Logging
Every tag configured in Voltrus is automatically logged to the historian at full resolution. When you add a Modbus TCP device and configure register polling, the data is stored time-stamped. No separate configuration required for logging—it just works.
Built-In Trend Viewer
Voltrus includes a web-based trend viewer that queries the historian and displays interactive charts. Operators can zoom from years to seconds, annotate events, and export data to CSV for analysis in Excel or Python.
Retention Configuration
Voltrus allows you to configure retention policies per tag or globally. Keep raw data for 12 months, roll up to 1-minute averages for 5 years, and hourly averages for 10 years. The system handles rollup automatically.
Export and Backup
The Voltrus historian stores data in a compressed binary format that can be backed up with a simple file copy. Export to CSV is built-in. Export to Parquet for analysis in Python/pandas is supported via the command-line tool.
Frequently Asked Questions
What is the difference between SCADA real-time data and historian data?
SCADA real-time data shows the current state of equipment at the moment it is polled. This is what you see on dashboards and operator screens. Historian data is time-stamped process data stored at full resolution for long-term analysis. Real-time data is overwritten with each new poll. Historian data is never overwritten—it builds a time-series record. The critical difference: real-time data tells you what is happening now. Historian data tells you what happened over hours, days, or months, allowing you to troubleshoot past events, analyze trends, and generate compliance reports.
Why do I need a historian if I have a SCADA system?
A SCADA system without a historian only gives you the present. When a problem occurs, you cannot see what led up to it. When a regulator asks for last month's chlorine residual data, you cannot provide it. When management wants to analyze pump run times to plan maintenance, you have no historical baseline. A historian converts your SCADA system from a monitoring tool into an analysis platform. The storage cost is trivial compared to the value of having complete historical records for troubleshooting, compliance, and optimization.
What is the difference between a historian and a database?
A historian is a specialized time-series database optimized for process data. General-purpose databases (PostgreSQL, MySQL) store data in rows with indexes. Historians store data in time-ordered sequences with compression. For the same data volume, a historian typically uses 10-30% of the storage space of a relational database and queries 5-10x faster for time-range queries. Historians also handle data rollup, downsampling, and interpolation automatically. You can build a historian on top of PostgreSQL (using TimescaleDB or similar extensions), but purpose-built historians include these features out of the box.
Should I build a historian with InfluxDB and Grafana or buy a SCADA system with built-in historian?
Building a DIY historian with InfluxDB and Grafana works for hobby projects and simple monitoring. It gives you visualization and storage for free. However, it lacks critical production features: no built-in SCADA protocol drivers (you need middleware to bridge Modbus/DNP3 to InfluxDB), no alarm management, no redundancy, no audit trails, and significant maintenance overhead. A commercial SCADA system with a built-in historian includes protocol support, alarms, redundancy, and support in one integrated package. For critical operations, the integration and support justification usually outweighs the cost savings of DIY.
How long should SCADA historian data be retained?
Retention strategy balances storage cost with analytical value. A typical tiered approach: keep raw data (1-5 second interval) for 6-12 months for detailed troubleshooting; roll up to 1-minute averages for 2-5 years for trend analysis; roll up to hourly or daily averages for 10+ years for compliance reporting. Critical compliance data may be retained permanently. Storage is cheap—1 TB of disk costs less than $50 and can store years of data from hundreds of points. The tradeoff is query performance: very long histories require rollup to remain responsive.
Historian Included, No Extra Cost
Voltrus logs every tag automatically to the built-in historian. Interactive trend viewer, configurable retention, export to CSV. $249 lifetime, unlimited tags.
Explore Voltrus SCADA