Practical Guide for Validation of Process Analyzer Systems According to the International Standards

Process analyzer validation according to ASTM standards is pivotal for ensuring operational efficiency and compliance. However, the intricacies within these standards, coupled with the variance in actual maintenance activities, often lead to ambiguity and inefficiency. This white paper written by Dirk Horst and Martin van Burgh endeavors to provide a pragmatic approach to process analyzer validation, focusing on transparency, control, and reproducibility.

In this comprehensive guide, we elucidate essential procedures for continual validation and analysis performance of online process analyzer systems. By distilling practical insights and bypassing superfluous complexities, we aim to equip stakeholders with accessible tools and methods for optimizing process analyzer validation.

Abstract

During a long period of working as QMI Engineer for Shell Global Solutions, invaluable experience has been gained in the field of performance monitoring of online analyzer systems by regularly determining whether they meet the requirements of their intended purpose.

The purpose of this article is an attempt to clarify important issues and make more transparent the commonly used procedures as applied for tracking the Continual Validation and Analysis Performance of online Process Analyzer Systems, not limited to Process GCs, as described in the ASTM D3764 and ASTM D6299.

Because several parts in these standards are not directly involved with the actual regular maintenance activities for continuous performance monitoring, such as preparation of Multi- Variable Laboratory Reference Materials, Statistical processing of data, these parts of the normally applied Standards are not discussed here. This without compromising its intended goal for optimal operational efficiency of the online analyzers, by using proven methods.

In order to improve the transparency and practical understanding of the Standards and make the monitoring process more attractive, simple tools are proposed as some have also been mentioned before in ASTM procedures.

Several reputable suppliers have developed automatic maintenance and performance monitoring software to help companies achieve efficient process analysis through online packages that are available under various names like “AMADAS’, “AIM” and others. (See the end of this article for some suppliers). These proposed tools have already proven to be of great value in implementing and automating the following procedures for most of the installed types of online Process Analyzers. Earlier the proposed procedures have been successfully applied (at several Shell sites) manually using simple Excel/Spreadsheet tools, as will be presented.

Goal

The aim is to give all who are involved with the routine tracking of online process analysis performance an optional tool and method for improvement, while providing an easy to understand (‘Keep It Simple’) guide, making it also more attractive to apply the proposed methods. This while they are still based on the professional, scientifically based practices as described in the Standards.

This procedure, using the generally applied Individual Observation (I) Control Chart, in combination with Statistical Decision rules and the proposed additional practices, has proven to be very useful and offers great insight view into the performance of all types of online, as well as off-line, process analysis systems.

In addition, this procedure also effectively assists by using the proposed “Reproducibility Rate” in determining and reporting the so-called “bad-actors” (process analyzers) that operate below expectations and therefore require excessive maintenance efforts.

Obviously, these discussed methods are essential for accurate performance of Critical Analyzers, as applied for Custody Transfer, but certainly also offer the tools for more efficient Process Control.

Scope

Practices include the following activities and information concerning:

• Analyzer Initial Calibration,
• Validation by Reference Material providing the Accepted Reference Values – ARV’s,
• Continual Validation using Control Charts,
• Performance Tracking by evaluation of historical data using the “Reproducibility Rate”,
• Notes and clarifications of the proposed practices concerning the Standards ASTM D3764 and ASTM D6299.

Definitions and terminology

A summary of the definitions and terminology used in this paper are the same as given in the standards and are also available here below on page 10.

Remarks concerning the control chart set-up procedure

The listed procedural sequence is essential for setting up a validation and performance monitoring system like the initial Analyzer Calibration during a Factory and Site Acceptance Test (FAT and SAT).

Once the process analyzer system has proven to be suitable for the job, it needs to be Initially Calibrated under stable normal operating conditions, meaning under Statistical Control.

The process analyzer system is calibrated using and agreed Primary Test Method in cooperation with the Laboratory (see for more applied options Chapter 6 of ASTM D6299)

• The purpose of this Initial Calibration is to exclude Systematic Errors like Bias and Linearity errors of the analyzed ARV result.

For the Calibration procedure, Certified Reference Material (CRM) is used that is prepared and supplied by recognized and certified Laboratories only.

Alternatively, these references may also be replaced by theoretical or scientifically established value, while this Reference Material should contain all of the Accepted Reference Values.

Methods for preparation of these Certified Reference Material should be traceable to International Standards while the chemical properties and component concentrations must resemble the actual process sample concentrations as far as possible!

The CRM’s used for the initial calibration may also be used as regular Validation Reference Material.

Procedures for developing process analyzer validation Reference Material are given in Chapter 6 of the ASTM D6299.

Suppliers of the Reference Materials should, in addition to guaranteeing the component concentrations supplied, must also guarantee their Stability, which is a critical requirement for their application as base reference in Control Charts, the ultimate tool for historical performance tracking

The capacity of each reference material supplied must be sufficient to cover a useful long period of monitoring the analyzer system, ranging from at least of six months to one year.

It is good practice is to record a reminder in the Maintenance Log Chart of the process analyzer for timely ordering of the Reference Material based on the total of the normal amount consumed.

Note that while it is common practice to introduce the validation reference sample close to the analyzer system, it may be mandatory that the introduction of the reference material should also include the entire sampling system, as required for Continuous Emission Monitoring Systems (CEMS).

• For a Multi-Component Analyzer, such as a Gas Chromatograph, one must create individual CCs for each measured component or property, as the Standard Deviations may be different from each other.

The control chart tool

A Control Chart (CC), initially developed by Walter Shewhart is a statistical process control chart, widely recognized for process quality control analysis, to understand how a process or measurement system may change over time.

It provides a unique tool to regularly check whether a process analyzer system is still working within pre-defined limits while the entire system is under statistical control.

The purpose of this audit is to regularly confirm (validate) that the analysis results of the ARVs are still within pre-determined limits.

Note that repeated results of a physical measurement process usually, when plotted under normal stable conditions, form a “Normally Distributed”, Gaussian curve which is assumed to be valid for this Control Chart procedure.

Did you notice?

It makes no sense to correct for random errors, which unfortunately appears still seem to happen, simply because the lack of understanding!

Description of the applied i-type control chart

The main elements of the control chart (see figure 1 below)

The Control Chart is a visual time series chart illustrating data points and process analyzer validation results collected at specific time intervals.

Determining the warning and control limits as used for corrective action

To be able to make decisions for any corrective action, we need limits that are based on the actual performance of the analysis system, meaning that these limits represent the best achievable accuracy for this particular analysis system under normal operating conditions.

• Depending on the criticality of the analysis, one is free to set different limits (such as e.g. per advice by the supplier) but then the following tracking procedures, based on actual performance, do not apply.
• In order to determine the applied Warning and Control limits in a Control Chart, we first need to repeatedly analyze the Reference Material (the ARV), such as for example a Calibration Gas mixture for a Process Gas Chromatograph (PGC) to determine the Variance that shows the spread of the results obtained.

Important notes

Note the important remark in ASTM D6299 A 1.6.3 concerning the ARV and AIM value Quote: “so the average difference between the check standard results and the Accepted Reference Value (ARV) is statistically indistinguishable from zero”.

In other words, the ARV value is assumed to be free of errors except for the agreed uncertainty, and can therefore be used as the Reference Value for Control Charts.

If we were to get these results in too short period of time, we would get the Short-Term STD that results in limits that are too narrow!

For latter reason, we must use the Long-Term Standard Deviation defined by ASTM as the “Historical Standard Deviation” as obtained in the following procedure.

How to validate your process analyzer step-by-step guide

1. After the initial acceptance of the analysis system as fit-for purpose, a minimum of 15 analysis results should be collected. This is done over a period of two weeks, when the stable analysis system continuously analyzes the normal process samples. Therefore, on a daily basis, the online analyzer is temporarily switched from the Process sample to the Reference Material (sample with all ARV concentrations) to collect “As-Found” results.
2. It is strongly recommended to collect a minimum of 5 analysis results, the so called “as- found” values, and to record only the stabilized results.
3. In the rare instance that “Outliers” appear in the dataset, it may be required to apply (automatic) outlier detection procedures in accordance with the Standards, while further investigation is strongly recommended!
4. The “as-found” process analyzer validation results only need to be recorded (manually or automatically) without any modification to the analysis system or any correction to the obtained data!
5. With the historical data collected, the Sample Standard Deviation (s) can be calculated using the above indicated formula
6. Graphically construct the Control Chart as shown in the figure above and draw the horizontal lines for the Control and Warning limits – 3 x s , – 2 x s , + 2 x s and + 3 x s respectively.
7. Then the running analyzer (or any measuring system) can then initially be validated frequently while simply recording all results for later evaluation.
8. The evaluation and any resulting corrective action are subject to the following (generally applied) Statistical Decision Rules that continuously review the historical data obtained from the previous validation results. See for an overview of the rules and follow-up in the table below.
9. The outcome of the Statistical Decision rules may or may not require corrective action but should always be recorded in the “Maintenance Log” section of the particular analyzer system and AMADAS software which may be part of the automatic performance monitoring software package or even in a simple electronic data base.

The reproducibility rate – a simple but effective tool for process analyzer validation!

The latter procedures are similar in the International Standards, while application of Control Charting is widely accepted and an integral part of most of the software packages available.

However, the features that track historical performance are often not very transparent and it would be very useful to make use of a “performance score value” that would make it much easier comparing online analyzers running on particular applications.

The “Reproducibility Rate” turns out to be such a very efficient useful score value!

• Analysers running the same analysis configuration may show different scores making it simpler to identify “bad-actors”. For latter reason, based on long term practical experience, it is highly recommended to apply this one of the best and simple evaluation features to track the actual performance of any analysis- or measuring-system, the so-called “Reproducibility Rate”.

A practical rule is that 95% of the previous process analyzer validation results should fall within the Warning Limits as this actually represents the repeatability range and so looking back on a number of previous results, we can evaluate the performance and express it in the Reproducibility Rate as follows.

How to determine the reproducibility rate?

With the above equation the Reproducibility Rate (RR) can easily be calculated.

A practical rule is that 95% of the validation results should fall within both Warning Limits as this represents the 95% repeatability range (Confidence Level).

To achieve this, at least 19 of 20 results should fall within the set Warning Limits.

Since the number of required corrections /adjustments very clearly depend on the initially set Warning Limits (+/-2 x Standard Deviation), they have a significant influence on the maintenance efforts required to achieve the highest accuracy and so both, limits and maintenance efforts need to be in balance. For latter reason we may later evaluate and decide to make small (acceptable) corrections to these limits to achieve an RR of 95%.

Setting the optimal process analyzer validation frequency

Only after the analyser has run more than 20 validations on the process sample for a period of time can an evaluation of the limits been be performed using the Reproducibility Rate.

At the same time, it will become clear from the achievable accuracy of the results what the optimal validation frequency is, as this is a balance between the number of required validations with necessary corrective actions taken needed to maintain the 95% RR.

For bad performing systems it will be difficult to maintain the 95% RR that will require a high process analyzer validation frequency (high maintenance efforts) and so one may to decide to adjust for a lower frequency by slightly adjusting (widening) the initial determined limits, while accepting the lower accuracy performance!

Process analyzer validation summery and conclusions

The reason for applying of the above-proposed validation methods and tools suggested above for performance monitoring of online process analyser systems using the widely applied Control Charting is because the special ASTM standards are hardly used in practice.

Apparent reason for the latter is that the standards do contain too much complex unnecessary information, not easy to apply for QMI maintenance personal.

Based on practical experience gained in the field at a number of Shell locations with the proposed procedures and periodic reporting, the interest has not only been aroused by QMI

maintenance people, but in particular by providing management with transparent reports of the accuracy and performance of the installed analyzer systems.

Additional notes and clarifications of the proposed practices

1. Note that the AIM line in the Control Charts represents the Accepted Reference Value and is believed to contain not any systematic error.This although each value of the reference material will have some uncertainty as specified by the supplier of the Validated Reference Material.
2. As is clearly shown in ASTM D6299 Table A1.3, the Precision / Standard Deviation of the Measurement System is Level Dependent. For each ARV (accepted reference value) the variance needs to be determined, resulting in different Standard Deviation values as well as Warning and Control limits. For latter reason, using multiple overlayed Control Charts in a single display record can be very confusing and is therefore not recommended.
3. As long as the environmental conditions of the analyser system are constant and the reference material is not changed, the variance of the analyser system and the “site precision conditions” do not change. This means that the earlier obtained limits are stable for a long period of time. However, if any of these issues change, the variance and thus the limits must be re-established through a repeatability test.
4. For latter reason, it is a good practice to execute a repeatability test e.g., once a year.
5. Application of more than one Reference Material automatically implies that additional uncertainties are introduced. In case of different levels of concentrations for one component / property are used, such as for a linearity check of the measuring, it is advised to use ASTM D7235.

When starting a new control chart?

After a change is made to any of the following issues a new Control Chart should be started:

• Any essential change in the analysis system that has a direct impact on the analysis result,
• After exchange of the reference material with different ARV values which may also be required in case the measured process value has changed,
• After performing a new repeatability test resulting in new Standard Deviation.

Software for process analyzer validation to ASTM D3764 and D6299

Automatic on-line performance monitoring systems are available from several suppliers that deliver complete systems that may be fully integrated into existing the instrumentation infra- structure at your site:

• AIM Performance monitoring, predictive maintenance software & analyzer data collection by ASaP
• CalSys Analyzer Management software – by KROHNE
• AML Information Management (AMADAS) and Continuous Emission Monitoring Systems (CEMS) – by HINT
• AAIMS Advanced Analytical Instrument Management System – by Yokogawa
• AMADAS by Jaaji Software Technologies Pvt Ltd.

References

1. ASTM D3764
2. ASTM D6299
3. Linkedin, group members “Industrial online Analyser Systems”

Download Process analyzer validation ASTM D3764 and D6299 step-by-step guide PDF

• accepted reference value, n—a value that serves as agreed-upon reference for comparison and that is derived as a theoretical or established value, based on scientific principles,
• accuracy, n—the closeness of agreement between an observed value and an accepted reference value.assignable cause, n—a factor that contributes to variation and that is feasible to detect and identify.
• bias, n—a systematic error that contributes to the difference between a population mean of the measurements or test results and an accepted reference or true value.
• control limits, n—limits on a control chart that are used as criteria for signaling the need for action or for judging whether a set of data does or does not indicate a state of statistical control.
• precision, n—the closeness of agreement between test results obtained under prescribed conditions.
• repeatability conditions, n—conditions where mutually independent test results are obtained with the same test method in the same laboratory by the same operator with the same equipment within short intervals of time, using test specimens taken at random from a single sample of material.
• reproducibility conditions, n—conditions under which, test results are obtained in different laboratories with the same test method, using test specimens taken at random from the same sample of material.
• Definitions of Terms Specific to This ASTM Standard:
• analytical measurement system, n—a collection of one or more components or subsystems, such as samplers, test equipment, instrumentation, display devices, data handlers, printouts or output transmitters, that is used to determine a quantitative value of a specific property for an unknown sample in accordance with a test method.
• check standard, n—in QC testing, a material having an accepted reference value used to determine the accuracy of a measurement system.
• Discussion—A check standard is preferably a material that is either a certified reference material with traceability to a nationally recognized body or a material that has an accepted reference value established through interlaboratory testing. For some measurement systems, a pure, single component material having known value or a simple gravimetric or volumetric mixture of pure components having calculable value may serve as a check standard. Users should be aware that for measurement systems that show matrix dependencies, accuracy determined from pure compounds or simple mixtures may not be representative of that achieved on actual samples.
• in-statistical-control, adj—a process, analytical measurement system, or function that exhibits variations that can only be attributable to common cause.
• quality control (QC) sample, n—for use in quality assurance programs to determine and monitor the precision and stability of a measurement system, a stable and homogeneous material having physical or chemical properties, or both, similar to those of typical samples tested by the analytical measurement system. The material is properly stored to ensure sample integrity, and is available in sufficient quantity for repeated, long-term testing.
• site precision (R8), n—the value below which the absolute difference between two individual test results obtained under site precision conditions may be expected to occur with a probability of approximately 0.95 (95%). It is defined as 2.77 times the standard deviation of results obtained under site precision conditions.
• site precision conditions, n—conditions under which test results are obtained by one or more operators in a single site location practicing the same test method on a single measurement system which may comprise multiple instruments, using test specimens taken at random from the same sample of material, over an extended period of time spanning at least a 15-day interval.
• Discussion—Site precision conditions should include all sources of variation that are typically encountered during normal, long term operation of the measurement system. Thus, all operators who are involved in the routine use of the measurement system should contribute results to the site precision determination. If multiple results are obtained within a 24–h period, then it is recommended that the number of results used in site precision calculations be increased to capture the longer term variation in the system.
• site precision standard deviation, n—the standard deviation of results obtained under site precision conditions.

Our recommendations to learn more about process analyzer validation

To achieve optimal performance through an understanding of operating principles, possible interference and practical knowledge, you can attend the training course Courses Process Analyzers and Sampling System by none other than Dirk Horst himself.

In addition, Process Gas Chromatographs by Tony Waters is a guide to the fundamentals of applied gas chromatography and the process gas chromatograph, with practical procedures for design and troubleshooting This comprehensive resource provides the theory that underpins a full understanding of the fundamental techniques of gas chromatography and the process analyzer.

Analytical Solutions and Products employees are trained by Tony Waters and are very familiar with his and the work of Dirk Horst.

Contact Our process analyzer validation specialist today

For personalized assistance and further process analyzer validation inquiries, please contact our specialist at ASaP. We are here to help and provide the support you need.