How does the auditor evaluate the results of audit procedures
4.1a, 4.1c, 4.1e, 8.1b, 8.1c, 8.2.2, & 8.2.3 in ISO 9001:2008; & 4.5.1 & 4.5.5 in ISO 14001:2004; and 4.5.1 & 4.5.5 in OHSAS 18001:2007
PURPOSE - A how-to guide for conducting an audit and completing an Audit Checklist.
M easuring the environmental management system [EMS], occupational health and safety management system [OHS] and/or quality management sysdefinitionprocesses to demonstrate the ability of the processes to achieve planned results),
Verifying the EMS, OHS, and/or QMS:
is who we are (planned arrangements)
is what we want to be (requirements established by the organization)
fulfills the sections of ISO 9001:2008 [QMS], ISO14001:2004 [EMS], and/or OHSAS 18001:2007 [OHS] called for by the applicable audit report or audit schedule
E nsuring the system is driving our behavior (effectively implemented and maintained) in an objective and impartial manner (which is what this procedure does)
SCOPE - All activities related to auditing of the management system at the City of Dallas.
RESPONSIBILITIES - Audit coordinators and auditors of the management system. Additional responsibilities and authority may be included below.
PROCEDURE (process flow with associated notes shown below)
Audit - A planned and documented audit performed in accordance with procedures or checklists for the intended purpose of verifying applicable elements of an ISO 9001 (QMS), ISO 14001 (EMS), and/or OHSAS 18001 (OHS) have been developed, documented, and effectively implemented in accordance with specified requirements.
Audit Plan - Typically an audit report based on the applicable audit requirements in ISO 9001, ISO14001, and/or OHSAS 18001 for the activity/area being audited and the Audit Report Summary, with additional questions/issues that are to be verified included in or attached to these documents as needed to ensure objectivity and impartiality. May also be a marked-up copy of the procedure/process documentation, identifying evidence to be collected to verify conformance.
Auditor - A qualified and trained individual who is authorized to perform specific audit functions (may be EMS, OHS, QMS, or some combination therein) under the direction of a lead auditor.
Audit Coordinator - Person with responsibility/authority for scheduling audits, selecting Auditors (ensuring objectivity and impartiality), and ensuring issues raised are effectively addressed.
Effectiveness - The evidence, including the relationship with inputs and outputs for the process, shows the process is working, driving performance, and supporting the organization's policy, objectives (including fiscal responsibility and sustainability), and compliance with requirements (laws, regulations, etc.).
Finding - An issue needing resolution. It could be an actual problem (something requiring corrective action), a potential problem (something requiring preventive action), or any other opportunity for improvement (including those making us better and/or helping us be more fiscally responsible). These "problems" are also known as non-conformances or deficiency or lack of conformance with any element of the management system (both quality and environmental). All non-conformances must be formally resolved to assure effective correction of the observed condition and the adoption of system improvements or preventive measures to reduce or preclude the likelihood of recurrence. Types of findings are:
MAJOR = The evidence shows the problem to be systemic (very big or bad) and/or requirements from the applicable standard(s) are not addressed or adhered to;
MINOR = The evidence shows a problem, in need of attention, but not one where the system is broken down (simply needs a little touch-up) and/or a requirement or two from the applicable standard(s) are not completely addressed or adhered to; and
COMMENT, OPPORTUNITY FOR IMPROVEMENT, or OBSERVATION = May be a praise or may be pointing out things that could use a little work (correction, preventive action, or opportunities for improvement).
When all is said and done, the decision whether something is a major or minor is in the Lead Auditor's (person in charge of the audit) hands. The tendency is to use "the benefit of doubt" (things start as a minor and escalate as supported by evidence) as the rule of thumb. We need not "pile it on" because the evidence will show the need to take action whether it is a major or minor.
Internal Auditor - A qualified and trained individual (see Internal Auditor criteria), who performs EMS, OHS, and/or QMS audits of City of Dallas departments and facilities, to report non-conformances and observations, and to evaluate the adequacy of corrective and preventive actions, reporting audit findings to a Lead Auditor.
Lead Auditor - A qualified and trained individual (via a certified Lead Auditor and receives a certificate of completion as a Lead Auditor or a Certified Lead Auditor certification from a Certification Body), who is authorized to plan, organize, and direct EMS, OHS, and/or QMS audits of City of Dallas departments and facilities, to report non-conformances and observations, and to evaluate the adequacy of corrective and preventive actions.
Noncompliance - Evidence indicates the organization is not complying with a regulation, rule, or requirement where compliance is mandatory (i.e. law, corporate policy, etc.).
Nonconformance - Evidence indicates the actions by those fulfilling a process and the information in supporting documentation do not conform with one another and/or requirements outlined in a standard (i.e. ISO 9001, ISO 14001, OHSAS 18001, etc.).
Objectivity and Impartiality - An expectation of both Auditors and the process they employ. To be objective and impartial means to let the evidence speak for itself. Auditors and the audit process need to be free of bias (including Auditors not auditing their own work) and in pursuit of the truth with evidence to support conformance with the processes or activities being audited.
COMPETENCY/QUALIFICATION OF AUDITORS USED. Persons trained to this process or an auditor training course that covers material comparable to that included in this procedure may be assigned to perform an audit, unless the audit will involve their own work. If the audit involves their own work, someone else will perform the audit. This ensures the auditor will maintain objectivity and impartiality. When training to this process or any other auditor training program, familiarity with ISO 9001:2008, ISO14001:2004, and/or OHSAS 18001:2007 (as applicable) is important.
Internal Auditor Criteria:
1) Attend 20 hours Internal Auditor training class when scheduled.
2) Conduct six (6) internal audits: two (2) observation audits with a Lead Auditor; four (4) audits conducted with a Lead Auditor observing the Internal Auditor conducting the Internal Audit and completing the audit checklist within 72 hours of completing the Internal Audit; and the Lead Auditor completing an Internal Auditor Evaluation on the Internal Auditor.
3) Lead Auditor of the Internal Auditor accesses the following: (a) Arrive at scheduled time
(b) Arrived prepared
(c) Level of participation
(d) Level of performance
(e) Audit report skills
(f) Knowledge of standard
4) Internal Auditor must score a minimum of 65 to be considered a competent Internal Auditor.
Internal audits are scheduled by the COD Management Representative. These audits are scheduled based on how well the area to be audited is doing (status), the effect a problem would have on the area if it was allowed to go on until a problem was detected (importance, risks, and/or significance) and how well the area did on the last audit. At a mimiumum, audits will be conducted on an annual basis and at a maximum audits will be conducted quarterly. The audit criteria are the clauses or sections of the applicable standard and/or the documents defining the activities or processes being audited. Scope tends to be based upon the sections of the applicable standard in use, but may be based upon a defined area or process.
While maintaining a "big picture" view of the process, the audit team (may also be an individual auditor) develops an understanding of the service or product provided by the process and the inputs from other processes. This perspective helps develop an audit plan focused on performance, instead of simply verifying a document is being followed.
Work with management and those responsible for environmental and/or occupational health and safety requirements applicable to the area being audited to determine if there are compliance or performance considerations needing attention. If environmental or occupational health and safety performance/compliance requirements are applicable, add these requirements to the plan.
Based on the processes/areas determined to be part of the audit, request copies of the applicable documents. These documents may include:
The environmental, occupational health and safety, and/or quality manual,
Procedures, work instructions,
or forms supporting the activity/area, and
Peripheral documents (standards/regulations, specifications, procedures, previous audit results, corrective/preventive actions issued since the last audit, etc.), as needed.
Document review and audit planning may be accomplished prior to and/or during the audit. The auditor uses the documents to formulate the questions to be asked and evidence to be reviewed. This plan may be a documented plan (an ISO 9001 audit checklist or an intregrated audit checklist. a highlighted document depicting questions to be asked/evidence to be reviewed, etc.) or may simply be a copy of the applicable standard, regulation, or management system document. In any case, the intent is for the auditor to familiarize himself/herself with the activities/processes to be audited and any applicable statutory/regulatory requirements associated with the process. The main thing to remember is, this is what might be seen not what will be seen (as it is possible that what is happening does not match the documents but the process is working). In other words, we need to stay open minded to allow the evidence to speak for itself.
The audit plan is the checklist, itinerary, or other such plan defining the evidence needing to be seen so we can come to a conclusion about the performance of the process. The Big Picture Audit report may be used to pull together requirements related to ISO 9001:2008.
Greet area personnel, helping them feel comfortable with the audit process and powering down your notes. (example: "Hi, I am here to perform an audit of ____ and I need your help. What I mean by that is I need you to speak on behalf of the process because you know what really happens. This will help us make sure the procedures are right. I will be taking notes along the way. My notes are not about you, but about the process. If there are things that need to be fixed or improvement opportunities, I will be writing those down so we can give management the information they need to take action.") REMEMBER: you (the Auditor) are not there to resolve any issues so do not make promises as to how or what will be fixed.ASK OPEN-ENDED QUESTIONS! We are looking for evidence related to performance and behavior. In order to find this and to make sure we are focused on what we need to see, open-ended questions help us ask the witness to show us how something works and what the results are. Also, make sure your questions are about the evidence that proves or disproves a process is working. While maintaining focus on performance and behavior, be sure to evaluate these things along the way (as the opportunity arises):
Familiarity with the policy and objectives (can the employee describe how they help the City of Dallas live up to the policy and objectives?).
Documents/records needed to support the process are available/retrievable, identifiable/legible, and are controlled as defined in document and records control procedures.
Personnel involved with the process appear to have received the education, skills, experience, or training needed to ensure the process is performing effectively (assessed along the way, with the conclusion reached at the end of the audit).
Resources need to ensure the policy, objectives, and activities or processes audited are and will continue to be effective/drive performance are in place (assessed along the way, with the conclusion reached at the end of the audit).CRITERIA:
J means what we said in a document is actually happening, meets the requirements, and is effective.
L means what we said in a document is not happening, the document does not meet all of the requirements, and/or is not effective.
K means we are not sure how we feel about what we see and probably need to ask additional questions to be able to come to a conclusion.
REMEMBER: Avoid rabbit trails (issues involving processes other than the one the Auditor is currently auditing), staying away from discomforts not directly related to the audit, unless it is obvious a nonconformance or ineffective condition related to the area or process being audited exists.
Make sure any findings are based upon facts not assumptions. Make note of the condition, powering down your notes so the witness does not think it is about him/her.
Remember, if you see something positive that is noteworthy, write it down. This is the only time you write someone's name on your audit report (unless their name is the sample, such as those you sample for training records). If your note taking skills are good enough to capture the evidence of conformance, that is great! At a minimum, make sure your results reflect what you audited (with the assumption being that no findings means the area/process is conforming and effective).
Ask additional questions or ask to see more evidence that will allow you to deal with your discomfort. Make sure they understand what you are uncomfortable with. This may help them with the answer. Be careful not to let your discomfort influence them into making process changes they do not need. REMEMBER: Make sure you are adding value, not imposing beliefs.
If your discomfort is something you feel will be an issue in the future, be sure to add it to your audit as a comment or observation. This will tell the next auditor what they should follow up on and may help area management recognize an opportunity for improvement, as any action they take would be preventive action. Work with affected personnel and area management to reach an agreement on the findings. While this may not be achievable (they do not like your results), try to obtain agreement, powering down the process along the way. If you still feel the finding is legitimate, there may be a need to agree to disagree with your findings continue to exist.
Once you are finished understanding the evidence of the process and whether that evidence reflects sound business practices, review the documents defining the process to see if the match up with what was seen. Any variation needs to be understood and worked through with area management to determine where there is a need to update the document (if it is found that the other things included in a procedure are not needed or there are additional steps that are not in the document but are needed) or a need to work with area personnel to ensure we are following practices deemed important.
The audit team or auditor summarizes the audit and any findings, presenting the recorded results to area management and/or the management representative. If the results are presented to the management representative, he or she reports the results to area management. The final audit report(s) are placed into Intelex by the 20th day at the conclusion of the audit.
A nonconformance was identified and will require corrective action?
A potential problem was identified and needs preventive action?
An opportunity for improvement was identified that could make us better (make more money or spend less money)?
If the issue(s) was able to be resolved before the end of the audit or it is believed the follow-up at the next audit will report on the results (Audit Report should be flagged to ensure follow-up), the audit may be closed out and the Audit Schedule is updated as needed (see note 1). Records (audit results, including any records of follow-up) are maintained as defined in records controls section of the Master Control Plan. The management representative summarizes the results of audits for management review.
Area management facilitates corrective action, preventive action, and/or any other improvement opportunity per Improvement Systems. This would include actions taken by area management (without undue delay) and follow-up of action taken. The records generated as a result of corrective/preventive action are maintained as defined in the records section of the Master Control Plan.
The audit team, auditor, or audit coordinator. work with area management to determine when to verify any actions taken (allowing enough time for effective implementation), closing out findings (reporting the results) that have been dealt with by auditing those issues in a manner similar to the original audit (following the methods established).
RECORDS - As defined in Internal Auditor Criteria and in Notes 16 & 17.Source: www.4eval.com