Abstract

Management of microbiological food safety is largely based on good design of processes, products and procedures. Finished product testing may be considered as a control measure at the end of the production process. However, testing gives only very limited information on the safety status of a food. If a hazardous organism is found it means something, but absence in a limited number of samples is no guarantee of safety of a whole production batch. Finished product testing is often too little and too late. Therefore most attention should be focussed on management and control of the hazards in a more pro-active way by implementing an effective food safety management system. For verification activities in a food safety management system, finished product testing may however be useful. For three cases studies; canned food, chocolate and cooked ham, the relevance of testing both of finished products and the production environment is discussed. Since the level of control of different processes can be largely different it is beneficial if the frequency of sampling of finished products and production environments would be related to the associated human health risk, which can be assessed on the basis of risk assessment and epidemiological data.

1. Introduction

Assurance of food safety moves more and more from end- or finished product testing to proactive food safety management. A food safety management system in a food processing company includes both control and assurance activities. Control activities are aiming at prevention or reduction of a food safety hazard and are typically related to product and process controls (Luning, Bango, Kussaga, Rovira, & Marcelis, 2008). Preventive measures are pre-requisite programs such as cleaning and sanitation, temperature control of the production environment, hygiene of the workers etc. elaborated in order to avoid contamination or outgrowth of microbial contaminants. Interventions in a production process are more focussed on reduction or even elimination of a certain contamination for instance by heat treatments. On the opposite, assurance activities in a food safety management system have the objective to provide evidence that products and processes are within set specifications. Examples of assurance activities are sampling, validation, verification, documentation (Luning et al., 2009). Therefore, the food businesses focus on the design and implementation of food safety management systems to guarantee food safety as demonstrated in a quantitative European study by Luning et al. (2015) and a Belgian study by Jacxsens et al. (2014). Since finished product sampling is valuable in some specific situations, for instance for traditional lot testing with hold/release or verification testing (see Buchanan and Schaffner (2015) for a good discussion on this subject), there is still much focus on finished product criteria and testing of finished products against set specifications. However, differences between criteria for products coming from production lines with different levels of control do not really exist, although more confidence could be given to a product from a well-managed processing line than from a batch of products that complies only with specific microbiological criteria without any information on process control.

Up to now, microbiological criteria given in Codex Alimentarius standards or in legislations (e.g. EC2073/2005), are mostly expressing the safety or hygiene standard of the product present on the market. However, in legislation, especially for environmental criteria, sometimes more long term process hygiene criteria are appearing (e.g. EC2073/2005). For efficient food safety control, it would be beneficial if this trend continues and would cover food safety management criteria. This would contrast with the use of microbiological criteria as sole measure of control, since due to statistical aspects and the heterogeneous distribution of contaminants (Jongenburger et al. 2012), the level of control of all achievable sampling plans is generally not appropriate.

One example where historical results can be used to reduce the frequency of end product testing already exists. In the EU legislation (EC2073/2005) for Salmonella analyses of minced meat, meat preparations and carcases, sampling can be reduced to fortnightly if satisfactory results have been obtained for 30 consecutive weeks or if the national or regional Salmonella control programme demonstrates that the Salmonella prevalence is low. If a similar approach could be applied for other food processes a reduction of microbiological tests could contribute to cost savings, still providing safe food to the consumers. The concept could be applied to all types of food processing operations. If past performance can demonstrate the effectiveness of prerequisite programs and the hazard control system (e.g. HACCP) at all steps of the food production process is implemented, from development and design to implementation and long-term monitoring, the frequency of microbiological testing could be reduced.

Following certain severe outbreaks, a risk management reaction may be to set new microbiological criteria. This happened for example after the EHEC/STEC outbreak related to sprouts in Germany, where new criteria were developed for this organism in this particular commodity (EU 209/2013, taken up into EC 2073/2005). However, as explained below, taking 5 samples of 25 g from a batch and showing they are all free from the pathogen, does not show that the whole batch is safe. In addition, the EU Regulation 209/2013 prescribes sampling of at least 0.5% of all batches of seeds and testing of 5 times 200 ml irrigation water. Although the latter test will make the detection of localised contaminations much more probable, the advised frequency is at least once a month, which again implies that occasional contaminations might easily be missed. The basis of the number of samples, frequencies and quantities of material to be analysed is often not easy to decide upon. Although these types of criteria do help to verify and detect occasionally deviations, they are not sufficient to guarantee full control (Jongenburger, den Besten, & Zwietering, 2015).

2. Sampling, validation, verification

By now, there is a general understanding that control of safety is only to a very limited extent supported by finished product testing. Good management should be based on evidence that hazards are well under control and that the interplay between initial levels of organisms, reduction, recontamination and growth is supplying a final level or prevalence of the hazard that is appropriate ( Fig. 1). Whether these phenomena are well under control needs to be based on solid information (validation) that can be partly based on sampling ( CAC, 2008). Especially data on initial levels and prevalence of microbiological contaminants in raw materials and the environment can be based on sampling, but is mainly done for investigating baseline data and general trends. For information on phenomena like reduction, survival, transfer and growth of micro-organisms along the production process or even the whole food chain, information from specific experiments (e.g. challenge tests), databases, scientific literature or predictive microbiology could be combined to determine proof of sufficient control.

Fig. 1. Overview of the relevant phenomena in food safety control, indicating the position of validation and verification.

Fig. 1.
Overview of the relevant phenomena in food safety control, indicating the position of validation and verification.

If in this manner, by validation, a process is shown to be under control, this can be verifiedby finished product testing at the food industry level and by epidemiology on governmental level. Neither absence of the microbial hazard in finished products, nor the lack of evidence for an epidemiological link, is proof that a process, and consequently the safety of food products, is under control. On the other hand, if finished products are not complying or if there is a strong epidemiological link, this can be an indication that a process is not under control. Therefore sampling as a verification activity may be a useful tool.

It can be stated that finished product sampling is a relevant part of the verification of a food safety management system, but that it is more the totality of information that provides the confidence, than the sampling only. A food safety dossier containing only abstract proofs of validation will not be sufficient without real field-data. However, finished product field data alone are not a proof of appropriate control either.

3. Which information is needed?

In many countries it is mandatory for food-manufacturers to work in accordance with Codex principles of Hazard Analysis Critical Control Point (HACCP) (CAC, 2003). Therefore, in most factories there should be historical records and other information available on the performance of applied processes and history of the product. This type of information is feeding the food safety management system and assuring that food safety in finished products is under control.

Depending on the time period for which production has been running the amount of historical data available will vary. Important records are those collected during monitoring at different steps in the production process (Table 1) and include process parameters such as time, temperature, pressure etc. Other important information covers results of microbiological tests on the prevalence or contamination level in the environment, incoming raw materials, semi-finished and finished products. This could be tests for food-borne pathogens (e.g. if following an investigation or root cause analysis testing for pathogens is deemed appropriate), but results on indicator or spoilage organism are often more relevant as the prevalence of pathogens normally is very low.

WhereWhatActivities and records
SuppliersEfficacy of their food safety management systemCompliance to an agreed standard of food safety management certification and efficiency of control measures records
Factory/Process stepsRaw materials (including packaging material)Monitoring or verification results on contamination of pathogens spoilage or indicator organisms based on a priori risk rating applied to supplier and raw materials
StorageTemperature atmosphere storage time
CCP monitoring results (e.g. heat process)Holding time and temperature
Process parametersTime to acidification pH drop cooling time etc.
Semi-finished productOccasionally microbial testing as verification: sampling plan and results
Prerequisite programsFactory environmentTest results on hygiene or pathogens from swab samples product residues or air quality
CleaningResults from verification of efficiency (visual inspections microbiological tests ATP-test etc.)
Verification of finished productsMicrobial testsSampling plan and microbiology results
Intrinsic propertiesVerification results on pH aW preservative concentration etc.
Extrinsic propertiesVerification results on modified atmosphere storage temperature etc.
After factory release during product shelf-lifeExtrinsic properties of finished productsVerification results on modified atmosphere storage temperature in-market supply-chain testing and product durability and stability testing

Initially, when only little data for a process (line) is available, finished product sampling is useful as a verification tool to guarantee that the product and process meet set specifications. Sampling of raw material will be important to identify relevant hazards and to show how effective the inactivation during processing is. The prevalence in raw materials may be high(er), but this is not necessarily a major problem if microorganisms are sufficiently eliminated during the production process. Increase or decrease in prevalence and concentration of the hazard in the raw material can be indicative of deviations.

Monitoring results at CCPs are vital as these are related to information on the variability and consistency of process parameters (critical and/or operational limits). Relevant records for thermal processes are for example pressure, temperature and holding time. For other processes times for acidification or cooling are central. Sampling of the production environment is relevant to show the potential for recontamination, especially in case of line start-up and packaging change-overs. As the environment is large and multiple sources of contamination routes are present (e.g. food contact surfaces, hands of personnel, air, water), the sampling plan should be well designed, targeting the most likely sources of recontamination, preferably close to the line where the product is not protected. Finished product testing is useful if there is reason to believe that the process is not well under control (e.g. CCP out of control or recontamination can occur), and as explained above, for verification of the whole food safety management system.

Testing of finished product samples can indicate whether microorganisms occur in the food chain and whether control strategies are effective, i.e. the efficiency of inactivation and recontamination control. However, if the prevalence and concentrations are expected to be low, the number of samples taken should be high in order to be able to gain useful information from the sampling. At the time the process and environmental pressure to recontamination is under control, it is more efficient to monitor the production process than to test finished products only.

Also it is important to have reliable suppliers consistently providing the required quality of raw materials. This can be achieved through audits which in particular are focused on the controls the supplier has in place to reduce the hazard in incoming raw materials. Records from supplier audits may be useful to document and assure consistent quality of raw materials supplied to the factory.

Analyses on incidents, recalls or consumer complaints should be carried out regularly and can lead to actions aimed at avoiding repetition of such events. When corrective actions are needed because critical limits have been violated the reason why the process went out of control must be established and measures to prevent recurrence should be implemented and documented.

4. Statistics of sampling

For sampling plans in which detection of positives is not accepted (c = 0), which is often the case for pathogens, the equation to determine the probability of detection is:

Pdetect(n,c=0,Pdefective)=1−n(1−Pdefective)

In words this equation can be explained as follows: The probability that one sample is not defective is 1 minus the probability of a defective. For all n samples to be not defective, the probability is this term to the power n. So (1-Pdefective)n is the probability that all nsamples are not defective. One minus this value is the probability that one or more of the samples are contaminated, so that the organism is detected in one or more samples.

This equation shows that the performance of sampling is often rather poor, definitely with a low rate of defective (i.e., contaminated) products (Table 2) and even when large numbers of samples are tested. Such very low rates of defectives are realistic for pathogens which occur infrequently and at low concentration.

Defective samples (Pdef)n = 1n = 2n = 5n = 10n = 15n = 20n = 30n = 60
000000000
0.010.010.020.050.10.140.180.260.45
0.020.020.040.10.180.260.330.450.7
0.050.050.10.230.40.540.640.790.95
0.10.10.190.410.650.790.880.96>0.99
0.20.20.360.670.890.960.99>0.99>0.99
0.30.30.510.830.97>0.99>0.99>0.99>0.99

From Table 2 it can be seen that even a sampling plan with 60 sample units has quite a low probability of detecting contamination rates of 1 or 2% as the probabilities of detection are only 45 and 70%, respectively. For example in a batch of 100,000 chocolate bars of which 1% (i.e. 1000 bars!) are contaminated with Salmonella, the probability that this rate of contamination would be detected with 60 sample units is only 45%, meaning that such a batch will go undetected in 55% of the cases. Obviously, Salmonellacontamination rates of 1 or 2% in chocolate would be unacceptably high. Also, the statistics presented in Table 2 assume that the contamination is homogenously distributed throughout the batch, and that Pdefective is equal for every sample unit taken.

5. Sampling and control in a typical food production process

Typically, in production processes, (1) the raw material undergoes inactivation to eliminate or reduce the level of microorganisms which are present, (2) recontamination from the processing environment may occur during the industrial processing and (3) growth may occur during transport and storage (either in a professional setting or at the consumer level) before the food is consumed (Fig. 2). The order of the inactivation, recontamination and growth can be different as in this scheme. Microbial testing can be performed by sampling the food as raw material, during processing, and after processing or at the end of shelf-life in case of perishable foods. Also, the production environment can be sampled and tested to identify the potential for recontamination.

Fig. 2. Flow chart representing the processes that should be under control for safe food production, and their typical place in food production. The concentrations in the products can change due to reduction, recontamination and growth, as explained in the main text.

Fig. 2.
Flow chart representing the processes that should be under control for safe food production, and their typical place in food production. The concentrations in the products can change due to reduction, recontamination and growth, as explained in the main text.

Fig. 2 represents a general flow chart of these typical elements of a food production process from raw materials to consumption. It is a strong simplification because inactivation, recontamination and growth can occur at several steps of the process. The flow chart stresses that if inactivation eliminates microorganisms and recontamination is prevented, production is under control. If microorganisms are still present at low numbers, prevention of growth (e.g. by short storage times at low temperature) will keep the level low until consumption.

Changes in concentrations of microorganisms can be expressed mathematically. If Nrm is the concentration in the raw material, Nf1 is the concentration in the food product after inactivation, Nf2 is the concentration in the food product after recontamination and Nend is the concentration in the finished product, they relate as:

View the MathML source

Nf2=Nf1+Rec

View the MathML source

where log Red is the log reduction by inactivation, Rec is the concentration increase due to recontamination expressed as cfu/g, cfu/ml or cfu/cm2 (NOT on a log scale) and log Growth is the log increase due to growth. Note that if the food product is not contaminated after recontamination, Nf2 equals zero and growth is not possible: Nendwill be zero too.

Clearly, if log Red is high, Rec is low and log Growth is low, the process is well under control and the concentration in the finished product will be (very) low. If processes that should ensure inactivation and prevention of recontamination and growth are effectively controlled and show excellent historical records, finished product testing can add little to the control of safety.

6. Case studies on products

In this paper, the production processes of three different types of food products are presented as three case studies: canned food (specifically considering Clostridium botulinum), chocolate (specifically considering Salmonella spp.) and cooked sliced ham (specifically considering Listeria monocytogenes). The processes have different characteristics in terms of reduction, recontamination and growth and therefore put different demands on food safety control and management. For the three case studies the role of the finished product testing is evaluated and it is hypothesised that identification of the impact of process steps that may lead to reduction, recontamination of growth can be used as tool to assess the importance of sampling and control in the process. This is studied by analyzing the available data.

 7. Canned products

For foods that are sterilised in hermetically sealed cans (Fig. 3), generally a minimal F121°C value of 3 min (at the slowest heating point) is used to guarantee sufficient reduction of C. botulinum spores (for low-acid products). For an often assumed D121°Cvalue of 0.21 min (Bean et al. 2012), this results in a 14.3 log reduction. This means virtually absence of the organism. If initially 100 spores of the organism would be present in a can, only 1 in 1012 cans would have a survivor present (Bean et al. 2012). This is in the same order as the yearly world can production. If different initial levels are assumed of course a different defect rate is obtained, however with >12D processing, very low probabilities of survival are obtained.

Fig. 3. Schematic process for the production of a canned food (red = reduction, purple = recontamination, blue = growth). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3.
Schematic process for the production of a canned food (red = reduction, purple = recontamination, blue = growth). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

For acid foods lower F121°C values can be accepted, resulting at these lower pH values also in a >12D reduction, since the D121°C at lower pH values is smaller than at neutral pH (and additionally C. botulinum is not able to grow at pH < 4.6 ( Bean et al. 2012)). In order to also control food spoilage often producers use higher F121°C values, especially for products where the intrinsic properties can increase the heat resistance of spores like presence of fat.

If the can is perfectly sealed, no recontamination can occur, and the number of organisms is totally determined by the level of surviving organisms. However, if the sealing is not perfect, generally other contaminants than C. botulinum will re-contaminate the product. As the environment where this contamination will take place will not be anaerobic, growth of Clostridium spp. as result of recontamination is not realistic, although a probability of spoilage is pertinent.

Finished product testing is in this case not at all realistic (even if tests would be 100% perfect, which they are not in practice), since even one survivor in 100,000 cans would not be acceptable. To be 95% sure that no can contains a surviving spore, 95% of cans need to be sampled. Then if all are negative it is known that at least 95% of the cans are free of the organisms, but only 5% of the cans are left. No assurance exists that the remaining cans are free of the organism and even a couple of cans with a survivingC. botulinum would be a calamity. Finished product testing therefore is clearly not of any use here, process control and robust procedures for establishing the scheduled process are the only useful food safety management options.

If a spoilage defective rate of 1 in 10,000 would be accepted, one could also test for surviving spores. However, to be 95% sure that the defective rate is below 1 in 10,000, 30,000 cans need to be sampled (as 0.999930,000 = 0.0498, so 5% chance that if a real defective rate of 1 can in 10,000 would be present, this would not be detected with 30,000 samples) (Table 3). This is again a non-realistic sampling plan. Alternatively, one could place all 100,000 cans on hold and see if certain cans expand or explode, indicating gas production. So with visible inspection huge amounts of cans can be tested non-invasively, and this may be achievable. But generally for cans all control is based on the validation and verification of the process. No finished product testing is carried out, at least not microbiological. In aseptic sterile products where recontamination is a possibility after processing, for example at filling, finished product testing could be useful.

ProductPtargetPdetectn = 1n for 95% detection probability
Canned food0.0000011per10000000.9999992995731
Canned food0.000011per1000000.99999299572
Canned food0.00011per100000.999929956
Canned food0.0011per10000.9992994
Chocolate coins0.0111per890.989265
Chocolate medallions0.29513per440.7059
Cooked ham low0.00891per1120.991335
Cooked ham average0.0321per310.96892
Cooked ham high0.0611per160.93948

A second method of verification is the use of epidemiological information. For this some information in RASFF, the European Rapid Alert System for Food and Feed (http://ec.europa.eu/food/safety/rasff/portal/index_en.htm), and the European Union summary reports (http://www.efsa.europa.eu/en/zoonosesscdocs/zoonosescomsumrep.htm) can for example be consulted. In the period 1998–2013, 39 hits can be found on “botulinum” in RASFF, with 3/39 hits on canned (and 0/39 on the word sterilised): C. botulinum in canned green beans from the United States 29/01/2008, Botulinum toxin in canned artichoke hearts from Peru 08/01/2007 and C. botulinum (type B) in Strawberries – stewed canned 24/09/1998. The cases are sparsely described and insufficient information provided to relate these to industrial canning. The overall number of reported cases within Europe is however rather low ( Table 4), and since it is a severe disease that also has a very short incubation time, underreporting will be much lower than with many other food-borne diseases.

YearOutbreaksOutbreaks with strong evidenceHuman casesHospitalisedDeaths
201265870
2011161035281
20109721201

8. Chocolate

In this case study we focus on the production of derived chocolate products starting from liquid chocolate such as pralines, medallions, chocolate bars etc. Although chocolate is considered as a relatively safe food product from a microbiological perspective (ICMSF, 2005), low levels of Salmonella spp. may provide a relevant risk of infection especially as these products are ready-to-eat and are often preferred by vulnerable sub-populations, for example children. Due to the low water activity of chocolate, no microbial growth can occur during the shelf life ( Podolak et al., 2010 and Tamminga et al., 1976). In addition the high fat content may protect the pathogen from heat and allow Salmonella to survive for long time in chocolate ( Komitopoulou and Penaloza, 2009 and Krapf and Gantenbein-Demarchi, 2010). Also the fat might protect the organism during stomach passage (Blaser & Newman, 1982). In outbreaks related to chocolate and derived products, low levels of Salmonella spp. have sometimes been found e.g. 4.3 cfu/100 g or 1 cfu/25 g ( Hockin et al., 1989).

Also via RASFF few occasions of Salmonella contamination are found by border control.Table 5 gives an overview of the presence of Salmonella in chocolate and derived products, reported by RASFF, between 01/01/2005 and 22/08/2014, with ‘cocoa and cocoa preparations’ – ‘pathogenic micro-organisms’ as keywords. According to RASFF, border control lead to blockage due to presence of Salmonella on imported fermented cocoa beans, stressing the importance of further processing of these raw materials towards ready-to-eat chocolate (https://webgate.ec.europa.eu/rasff-window/portal/?event=notificationDetail&NOTIF_REFERENCE=2012.ADL accessed 22/08/2014). Also search in the EU outbreaks investigations gave limited information on the food category ‘sweets and chocolate’ for the past years (Table 6).

Country involvedYearInvolved chocolate product
Ivory Coast2012Fermented cocoa beans
Romania2011Chocolate figurines
Slovak Republic2010Chocolate products
The Netherlands2008White imitation chocolate with yoghurt flavor
UK2006Chocolate bars

YearNumber of outbreaks related to ?sweets and chocolate?Total number of outbreaks reportedReference
20125 (1.4%)347http://www.efsa.europa.eu/en/efsajournal/doc/3547.pdf
201119 (6.7%)283http://www.efsa.europa.eu/en/efsajournal/doc/3129.pdfmainly associated with outbreaks in Poland linked to products from domestic markets and household preparations
20103 (0.9%)341http://www.efsa.europa.eu/en/efsajournal/doc/2597.pdf

Salmonella spp. can be present on the raw or fermented cocoa beans, which are not always processed and stored under the most hygienic conditions in the countries of origin ( da Silva do Nascimento et al., 2010). There are several potential biological control steps in cocoa processing (such as roasting and steaming) which would ensure elimination of Salmonella coming from raw cocoa beans ( da Silva do Nascimento, Brum, Pena, Berto, & Efraim, 2012). Prevalence of Salmonella spp. in the liquid chocolate is therefore not expected when processing towards liquid chocolate is well controlled.

The subsequent process steps of chocolate towards derived chocolate products cannot be relied upon to eliminate Salmonella. Although there are steps where pasteurisation temperatures may be reached, this will not be sufficient to kill Salmonella, which is known to be extremely heat resistant at the low water activities of chocolate ( Krapf & Gantenbein-Demarchi, 2010). Any pathogen introduced through cross-contamination from the environment during the process or through added ingredients like milk powder, nuts or dried fruits can be expected to be present in the final product.

Prevalence data on non-conforming units from a reported outbreak in 1985–1986 in Canada, 13 of 44 samples of chocolate medallions and 1 of 89 tested bags with small chocolate coins contained Salmonella spp. ( Hockin et al., 1989). In this example, the most contaminated batch will be detected with relatively few samples, whereas for the batch with low contamination rate, even large numbers of samples leave a high risk of no detection (Table 3). It is clear that detection of such a low prevalence by solely finished product testing is not feasible in practice and batch release based on only finished product testing is provoking a false sense of safety. Therefore, process and environmental control are necessary in the production of these types of products.

First strategy is related to process control: after processing of the cocoa towards liquid chocolate there are no steps in the production process which can eliminate Salmonella. The risk of Salmonella in chocolate products is more due to survival after recontamination than growth. Finally, chocolate production is a continuous process, where a niche with growth is possible. It will therefore be of upmost importance to control the temperature and humidity conditions along the chocolate processing line to avoid potential local increase of water activity and growth of Salmonella.

Secondly, introduction of other ingredients, such as milk powder, dried nuts or fruits to make derived chocolate products in a mixing step (Fig. 4), needs to be controlled as they can be a potential source of Salmonella spp. A search on RASFF for ‘nuts’ and ‘pathogenic microorganisms’ between 01/01/2005 and 22/08/2014 showed 47 cases, including raw pine nut, coconut, almond, hazelnut and pistachio nut, imported from various non-European countries, which were mostly not heat treated (through blanching, steaming or roasting). Therefore, ingredients and potential recontamination of the chocolate will be important but here again the fact that target levels are so low, results in that testing can never assure absence of pathogens (see Table 3). Control needs to assured by management of suppliers. Raw materials and ingredients can be grouped in certain risk categories requiring defined actions (which can rank from agreed specifications up to audits at suppliers of high risk raw materials at certain intervals). Furthermore, testing the ingredients for hygiene indicators such as Enterobacteriaceaecan be followed up as trend analysis. Typically the following guidance levels ofEnterobacteriaceae can be given to the supplied materials and ingredients >1000 cfu/g high risk, 100–1000 cfu/g medium risk, <100 cfu/g low risk.

Fig. 4. Schematic process for the production of a chocolate product (red = reduction, purple = recontamination. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4.
Schematic process for the production of a chocolate product (red = reduction, purple = recontamination. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

As a third strategy, environmental contamination needs to be under control. Prerequisite programs should include zoning, prevention of condensation, cleaning, equipment design, air treatment, etc. To make sure these measures are efficient, pathogen monitoring in the environment (e.g. building and infrastructure) and equipment/materials used in the chocolate production should be applied. Here the verification activity can be based on testing for hygiene indicators as Enterobactericeae and the detection ofSalmonella spp.

9. Cooked ham

This case-study is related to cooked boneless, formed ham. When arriving at manufacture, the pork muscles are cut and deboned; after the steps of brining, tumbling and forming in a plastic case, the product receives a heat treatment (Fig. 5).

Fig. 5. Process Flow chart of cooked ham (boneless, formed ham). Red = reduction, purple = recontamination, blue = growth. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 5.
Process Flow chart of cooked ham (boneless, formed ham). Red = reduction, purple = recontamination, blue = growth. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

During the thermal process, the internal cooking temperature (core temperature) is higher than 70 °C and the targeted pasteurisation value is more than 40 min (P70°C > 40). This thermal process is sufficient to completely inactivate L. monocytogenes cells potentially brought with the raw materials or introduced through the operational process steps before cooking ( Carlier, Augustin, & Rozier, 1996).

After the thermal process, the ham is sliced, packed and then stored under chilled conditions (Fig. 5). During mechanical slicing, recontamination of cooked ham byL. monocytogenes may occur ( Keskinen et al., 2008 and Sheen and Hwang, 2008). The sources of recontamination may be another piece of contaminated product (cross-contamination), or a biofilm surviving the cleaning and disinfection procedure. The level of recontamination has been estimated by analysing various literature sources (Table 7). On average, 3.2% of ham slices are contaminated when distributed in retail stores. These figures are in agreement with the investigations made by the EU Member States (EFSA, 2012): on average 5.1% of samples from pig-meat, cooked, ready-to-eat products collected at retail in 2011 and 2012 were L. monocytogenes positive (median = 3.5%, range from 0% to 26.3%). Obviously, having an average value of 3.2% means that some batches of cooked ham have a much lower prevalence (even some are not contaminated at all) and some other batches have a higher prevalence. It is important to bear in mind that analysing products collected at retail generates an uncertainty or even a bias due to potential L. monocytogenes growth and differences between sample and product unit weights.

Products and originSample size (g)No. of positive samples/no. Of samplesPrevalence (%)Reference
Luncheon meats USA2582/91990.89Gombas et al. 2003
Ham Brazil25Jan-651.5Martins & Germano 2011
Ham United Kingdom10040/9494.2Little Sagoo Gillespie Grant & McLauchlin 2009
Cooked ham Belgium2554/8796.1Uyttendaele De Troy & Debevere 1999
Prevalence estimatesMean 3.2

The level of L. monocytogenes contamination among slices of ham was estimated from Gombas, Chen, Clavero, and Scott (2003) by analysing the data obtained on luncheon meat from two US states (Maryland and California). The L. monocytogenesconcentrations were log-transformed before being processed with the RiskCumul function of @Risk (Palisade Corporation). The level of L. monocytogenes in contaminated ham was estimated to be low (median value: 0.074 cfu/g, mean value: 0.2 cfu/g) although the probability distribution was right-skewed (90th percentile: 31 cfu/g; 99 th percentile: 943 cfu/g) ( Fig. 6).

Fig. 6. Levels of Listeria monocytogenes contamination in positive samples of luncheon meat, data from Gombas et al. (2003).

Fig. 6.
Levels of Listeria monocytogenes contamination in positive samples of luncheon meat, data from Gombas et al. (2003).

Next, the ham slices are hermetically packed up to the time of its consumption. Consequently, there are no additional sources of recontamination. However, although the product is stored under chilled conditions, L. monocytogenes is able to multiply on ham ( Beumer et al., 1996, Hwang et al., 2012, Mataragas et al., 2006 and Zhang et al., 2012) and then potentially can reach hazardous concentrations for the consumer.

A search in RASFF confirmed this literature analysis. The search was done without any restriction of dates, with the keywords ‘Listeria” and “ham”, or, “Listeria” and “cooked”. From the initial 51 hits, after eliminating the duplicates and verifying the core text of the alert to focus as far as possible on the product sliced cooked ham, 20 cases were found. One case was not specified and therefore omitted. Since year 2004, there are regularly a few recalls from consumers due to ham products (Table 8). The notifications were either due to each company’s own tests (8) or official tests of products on the market (11).

YearNotification typeNotification basisSubjectAction taken
2004alertcompany's own checkL. monocytogenes (presence) in sliced hamproduct recall or withdrawal
2004informationofficial control on the marketL. monocytogenes (104CFU/g) in egg salad with hamprohibition to trade - sales ban
2005informationofficial control on the marketL. monocytogenes (<50 CFU/g) in ham from Cyprusproduct recall or withdrawal
2007alertofficial control on the marketL. monocytogenes (>100 CFU/g) in cooked ham from Italydestruction
2008alertcompany's own checkL. monocytogenes (presence/25 g) in vacuum packed paprika coated ham from the Czech Republicdestruction
2009informationofficial control on the marketL. monocytogenes (<10/25 g) in sliced cooked ham from Germanyno action taken
2009informationofficial control on the marketL. monocytogenes (presence/25 g) in ham from Irelandrecall from consumers
2009informationofficial control on the marketL. monocytogenes (presence/25 g) in sliced baked ham from Irelandwithdrawal from the market
2009informationofficial control on the marketL. monocytogenes (10 CFU/g) in vacuum packed ham from Polandofficial detention
2010informationcompany's own checkL. monocytogenes (2000 CFU/g) in cooked ham from Germanyrecall from consumers
2010informationcompany's own checkL. monocytogenes in rindless cooked ham from Francerecall from consumers
2010alertofficial control on the marketL. monocytogenes (5 out of 5 samples/25 g) in ham from Germanywithdrawal from the market
2011alertofficial control on the marketL. monocytogenes (4800 CFU/g) in cooked ham from Italydestruction
2011alertcompany's own checkL. monocytogenes (presence/25 g) in ham from Italywithdrawal from the market
2012alertcompany's own checkL. monocytogenes (<40 CFU/g) in chilled ham from Spainrecall from consumers
2012alertcompany's own checkL. monocytogenes (presence/25 g) in ham from Belgiumwithdrawal from the market
2012alertofficial control on the marketL. monocytogenes (presence in 4 of 5 samples/25 g) in spiced and cooked pork flank from Germanyrecall from consumers
2013alertofficial control on the marketL. monocytogenes (600 CFU/g) in frozen cooked meat product sliced in straight stripes from Poland with raw material from the Netherlandsdestruction
2014alertcompany's own checkL. monocytogenes (presence/25 g) in ham from Germanywithdrawal from the market

Testing raw materials for L. monocytogenes is not relevant as far as the HACCP plan is properly implemented and the process under control. The cooking step ensures the eradication of any L. monocytogenes brought by the pork meat, any other raw material or the environment during brining and tumbling process operations.

On the opposite, testing the finished product before manufacture release might be of interest. However, a sampling plan of 92 units (c = 0, n = 92 associated with for instance testing absence in 25 g) is required to guarantee that a defective piece of ham product is detected with 95% probability if the within batch prevalence is 3.2% (Table 3). Testing 92 samples of ham is much higher than the classical sampling plan recommended forL. monocytogenes in deli-meat. For instance, the European Commission (2005)recommends a sampling plan of 5 units of ready-to-eat refrigerated foods able to support the growth of L. monocytogenes. These 5 units do not prove that the prevalence in the specific batch is below 3.2%, but is able to detect gross deviations and gives on the long run verification of sufficient control.

Putting such microbiological criteria (c = 0, n = 92, absence in 25 g) in place will only make sense if the detection method is 100% sure, i.e. if there are no false negatives. With 50% of contaminated ham slices having a concentration lower than 0.074 cfu/g (Fig. 6), it might be analytically difficult (not to say impossible) to seek for a 100% effective detection method. Consequently, finished product testing is not the most efficient measure to control the microbial safety of cooked ham production. The most appropriate control measures are those associated with preventing recontamination byL. monocytogenes at the slicing steps. This can be achieved by implementing best practises on cleaning and operations of the factory environment around the slicer and packaging equipment.

In addition, an L. monocytogenes monitoring plan around the slicers (environment, equipment in contact with the product, floor … etc.) is recommended. To illustrate this latter point, an example of recommendations made by the New South Wales Food Safety Authority of Australia (2008) regarding L. monocytogenes in meat process environment is provided in Table 9.

Sampling plan in the environment:
It is recommended that at a minimum businesses operators sample five environmental sites for Listeria spp. monthly. Also it is indicated that testing must include samples collected before operations commence and during operations and cover all important work surfaces over time. Finally it is recommended that business operators maintain a consistently high level of hygiene and undertake sampling throughout the year.
Actions in case of positive sample found:
It is recommended that if a positive environmental sample is found the business operators immediately investigate the potential cause of the problem and initiate corrective action in accordance with its food safety program. Also it is mentioned that the suspect areas must be immediately identified cleaned and sanitised.
The business operators must ensure that food product is not contaminated and continue to review corrective action. This process enables to demonstrate adequate steps have been taken to minimise the risk of Listeria contamination of product.
Sampling plan following the corrective actions:
Whenever there is a positive environmental sample for Listeria spp. it is recommended that the business operators increase the frequency of environmental testing for instance from monthly to weekly testing and continue to test until the environmental swabbing program has achieved three consecutive negative sampling results. The purpose of increasing the testing frequency is to monitor the effectiveness of the corrective action that has been undertaken by the meat business. Moreover it is pointed out that where negative results are not being obtained the adequacy of the corrective action needs to be further reviewed and revised corrective action must be implemented.

10. Discussion and conclusions

In this paper the usefulness of finished product testing and the alternatives for validation and verification of food safety management in production processes are analysed on the basis of three examples (see Table 10).

Clostridium in a canned productSalmonella in chocolateListeria in cooked ham
Raw materialsMay be contaminatedMay be contaminatedMay be contaminated
Initial reductionVery effectiveVery effectiveVery effective
RecontaminationNegligiblePossible (contamination from ingredients or factory environment)Possible (from slicer or packaging machine)
GrowthIrrelevantNo growth but survival is possibleListeria can grow at chilled temperature
Possibilities for in process verification (HACCP in place and CCPs under control)Not needed when heating and sealing are carried out properlyEnvironmental samplingMonitoring ingredients for hygiene indicatorsEnvironmental sampling (e.g. slicers)
Epidemiological data: RASSF3 notifications in 16 years5 notifications in 10 years19 notifications in 11 years
Epidemiological data: EU summary reportsAbout 10 outbreaks per year not necessarily from industrially canned productsAbout 10 outbreaks per year related to chocolate or sweetsNo data found
Finished product samplingNot recommended (much less than 1 in 1000 products contaminated)Not recommended (less than 1 in 1000 products contaminated)May be recommended (about 3.2% of products contaminated)

For canned foods, sufficient reduction and prevention of recontamination can guarantee control of safety, and expected contamination rates are so small that finished product testing for pathogens is not realistic. Also for spoilage organisms finished product testing is not effective, unless a non-destructive test can be used, although even these tests could have limitations.

For chocolate, the processing of beans will be of importance to eliminate Salmonella in the liquid chocolate. Further in the production process of derived chocolate products, introduction via ingredients must be avoided. Therefore, selection of proper ingredients and supplier control needs to avoid introduction of Salmonella in derived chocolate products. Along the liquid chocolate stage, temperature and relative humidity shifts need to be controlled in order to prevent local increase of water activity. Finally, environmental control by zoning of production areas to avoid recontaminations needs to be elaborated and environmental sampling on indicators such as Enterobacteriaceae will be useful.

The most appropriate control measures to guarantee the microbial safety of the cooked boneless, formed ham product are those associated with preventing recontamination byL. monocytogenes at the slicing step. Indeed, cooked ham has the highest probability of recontamination (among the three products studied here) leading to a non-negligible rate of contaminated finished products at retail (0.89%–6.1% according the sources, seeTable 7). Therefore, an L. monocytogenes monitoring plan around the slicers (environment, piece of equipment, floor, etc.) is a must. On the other hand, testing raw material is not relevant as far as the HACCP plan is in place since the subsequent cooking step will fully inactivate L. monocytogenes. Likewise, the finished product contamination rate (prevalence and level) is too low to be efficiently detected by testing.

In all cases the contamination of raw material is effectively inactivated, except for the ingredients that can be added to the chocolate. This makes recontamination the most important contamination step in the process. The last case, cooked ham, has the largest probability of contamination at the endpoint and here verification by finished point sampling might be useful as growth of L. monocytogenes can occur even after the point of sampling.

In Europe the RASFF portal is an easy-to-use and well documented portal. Beside public health information (e.g. outbreaks due to C. botulinum), it provides also information on recalls from consumers due to non-compliance with microbiological criteria (e.g.L. monocytogenes). Both RASFF notifications and literature studies can help food manufacturers to identify hazards and evaluate the relevance of safety management measures and more precisely pinpoint where in the production line sampling procedures are effective measures.

The data obtained for the three case studies support the quick analysis on the basis of the flow chart, analysing the critical reduction, recontamination and growth steps (seeFig. 2). In canned food, inactivation occurs at high temperatures (Red is very high) and recontamination is unlikely (Rec is negligible), since the processing is carried out in a closed package. Growth is irrelevant, because the concentration in the can after inactivation Nf2 = 0. Finished product testing is not recommended. In chocolate, inactivation of microorganisms in raw materials during processing is effective (Red is high), but recontamination may occur (Rec is not negligible) in all following steps towards chocolate products. Growth is unlikely due to low water activity. Still, finished product testing is not recommended because contamination is rare. In cooked, sliced ham microorganism inactivation during cooking is highly effective, (Red is high), but recontamination may occur (Rec is not negligible). Growth is also likely to occur. Finished product testing may be recommended, at least for verification.

The case studies also show that a quick analysis is only a first step. Detailed information is needed to identify where environmental (cooked ham) or ingredient (chocolate) samples need to be taken. It is clearly shown that even when the hazard contamination rate is relatively high (e.g. in the chocolate case), a substantial number of samples still needs to be tested to identify a hazardous product.

The case studies confirm that assurance of food safety cannot be based on finished product testing. To assure safety, an efficient food safety management system must be implemented e.g. based on the HACCP principles and with proper pre-requisite programmes as described earlier and in Table 1. In the future it might be an option to provide food producers with “a license to produce” based on an accumulated record of evidence on control of the whole food chain, with testing used only for verification of the implemented food safety management system. This control should be focussed on those hazards that are identified to be relevant, for example in the HACCP study, and be based on the characteristics of the food (and its production), the characteristics of the potential pathogen, and the likely uses and foreseeable abuses by the consumer.

Acknowledgements

This work was conducted by an expert group of the European branch of the International Life Sciences Institute (ILSI Europe). The authors would like to thank Mr Christophe Dufour and Dr Elissavet Gkogka who were members of this expert group for their active contribution to this work. This publication was coordinated by Dr Alessandro Chiodini, Scientific Project Manager at ILSI Europe. The expert group was given the opportunity to receive funding from the ILSI Europe Risk Analysis of Food Microbiology Task Force. Industry members of this task force are listed on the ILSI Europe website at www.ilsi.eu. Current member companies are McDonald’s Europe, Unilever, Institut Mérieux, Arla Foods, Nestlé, Mondelez International and Mars. For further information about ILSI Europe, please email [email protected] or call +32 2771 00 14. The opinions expressed herein and the conclusions of this publication are those of the authors and do not necessarily represent the views of ILSI Europe nor those of its member companies.

References

    • Bean et al., 2012
    • D. Bean, F. Bourdichon, D. Bresnahan, A. Davies, A. Geeraerd, T. Jackson, et al.
    • Risk assessment approaches to setting thermal processes in food manufacture
    • ILSI Europe Report series 2012  (2012), pp. 1–40 http://www.ilsi.org/Europe/Publications/Thermal%20processing.pdf
    • Beumer et al., 1996
    • R.R. Beumer, M.C. te Giffel, E. de Boer, F.M. Rombouts
    • Growth of Listeria monocytogenes on sliced cooked meat products
    • Food Microbiology, 13 (4) (1996), pp. 333–340
    • Blaser and Newman, 1982
    • M.J. Blaser, L.S. Newman
    • A review of human Salmonellosis: I. Infective dose
    • Reviews of Infectious Diseases, 4 (1982), pp. 1096–1104
    • Buchanan and Schaffner, 2015
    • R.L. Buchanan, D. Schaffner
    • FSMA: testing as a tool for verifying preventive controls
    • Food Protection Trends, 35 (2015), pp. 228–237
    • CAC, 2003
    • CAC
    • General principles of hygienic practices (CAC/RCP 53-2003)
    • (2003)
    • CAC, 2008
    • CAC
    • Guidelines for the validation of food safety control measures (CAC/GL 69-2008)
    • (2008)
    • Carlier et al., 1996
    • V. Carlier, J.C. Augustin, J. Rozier
    • Destruction of Listeria monocytogenes during a ham cooking process
    • Journal of Food Protection, 59 (1996), pp. 592–595
    • EFSA, 2012
    • EFSA (European Food Safety Authority)
    • The European union summary report on trends and sources of Zoonoses, Zoonotic agents and food-borne outbreaks in the European union in 2010
    • EFSA Journal, 10 (3) (2012), p. 442 2597 http://dx.doi.org/10.2903/j.efsa.2012.2597
    • EFSA & ECDC, 2013
    • EFSA (European Food Safety Authority) and ECDC (European Centre for Disease Prevention and Control)
    • The European Union summary report on trends and sources of zoonoses, zoonotic agents and food-borne outbreaks in 2011
    • EFSA Journal, 11 (4) (2013), p. 250 3129 http://dx.doi.org/10.2903/j.efsa.2013.3129
    • EFSA & ECDC, 2014
    • EFSA (European Food Safety Authority) and ECDC (European Centre for Disease Prevention and Control)
    • The European Union summary report on trends and sources of zoonoses, zoonotic agents and food-borne outbreaks in 2012
    • EFSA Journal, 12 (2) (2014), p. 312 3547 http://dx.doi.org/10.2903/j.efsa.2014.3547
    • EU 209/2013 Amending Regulation (EC) No, 2073/2005
    • EU 209/2013 Amending Regulation (EC) No 2073/2005 as regards microbiological criteria for sprouts and the sampling rules for poultry carcases and fresh poultry meat. http://www.fsai.ie/uploadedFiles/Reg209_2013.pdf.
    • European Commission (EC), 2005
    • European Commission (EC)
    • Commission regulation no 2073/2005 of 15 November 2005 on microbiological criteria for foodstuffs
    • Official Journal of the European Union, L, 338 (2005), pp. 1–26
    • Gombas et al., 2003
    • D.E. Gombas, Y. Chen, R.S. Clavero, V.N. Scott
    • Survey of Listeria monocytogenes in ready-to-eat foods
    • Journal of Food Protection, 66 (2003), pp. 559–569
    • Hockin et al., 1989
    • J.C. Hockin, J.-Y. D’Aoust, D. Bowering, J.H. Jessop, B. Khanna, H. Lior, et al.
    • An international outbreak of Salmonella Nima from imported chocolate
    • Journal of Food Protection, 52 (1989), pp. 51–54
    • Hwang et al., 2012
    • C.A. Hwang, L.H. Huang, S. Sheen, V. Juneja
    • Effects of lactic acid on the growth characteristics of Listeria monocytogenes on cooked ham surfaces
    • Journal of Food Protection, 75 (2012), pp. 1404–1410
    • ICMSF, 2005
    • ICMSF (International Commission on Microbiological Specifications for Foods)
    • Cocoa, chocolate, and confectionary, in microorganisms in foods 6, microbial ecology of food commodities
    • Kluwer Academi/Plenum Publishers, New York (2005), pp. 467–479
    • Jacxsens et al., 2014
    • L. Jacxsens, K. Kirezieva, P. Luning, J. Ingelrham, H. Diricks, M. Uyttendaele
    • Measuring microbial food safety output and comparing self-checking systems of food business operators in Belgium
    • Food Control, 49 (2014), pp. 49–59
    • Jongenburger et al., 2012
    • I. Jongenburger, J. Bassett, T. Jackson, L.G.M. Gorris, K. Jewell, M.H. Zwietering
    • Impact of microbial distributions on food safety II. Quantifying impacts on public health and sampling
    • Food Control, 26 (2012), pp. 546–554
    • Jongenburger et al., 2015
    • I. Jongenburger, H.M.W. den Besten, M.H. Zwietering
    • Statistical aspects of food safety sampling. Annual
    • Reviews of Food Science and Technology, 6 (2015), pp. 20.1–20.25 http://dx.doi.org/10.1146/annurev-food-022814-015546
    • Keskinen et al., 2008
    • L.A. Keskinen, E.C.D. Todd, E.T. Ryser
    • Transfer of surface-dried Listeria monocytogenes from stainless steel knife blades to roast turkey breast
    • Journal of Food Protection, 71 (1) (2008), pp. 176–181Komitopoulou and Penaloza, 2009
    • E. Komitopoulou, W. Penaloza
    • Fate of Salmonella in dry confectionery raw materials
    • Journal of Applied Microbiology, 106 (2009), pp. 1892–1900
    • Krapf and Gantenbein-Demarchi, 2010
    • T. Krapf, C. Gantenbein-Demarchi
    • Thermal inactivation of Salmonella spp. during conching
    • LWT – Food Science and Technology, 43 (2010), pp. 720–723
    • Little et al., 2009
    • C.L. Little, S.K. Sagoo, I.A. Gillespie, K. Grant, J. McLauchlin
    • Prevalence and level of Listeria monocytogenes and other Listeria species in selected retail ready-to-eat foods in the United Kingdom
    • Journal of Food Protection, 72 (2009), pp. 1869–1877
    • P.A. Luning, L. Bango, J. Kussaga, J. Rovira, W.J. Marcelis
    • Comprehensive analysis and differentiated assessment of food safety control systems: a diagnostic instrument
    • Trends in Food Science and Technology, 1 (2008), pp. 1–13
    • P. Luning, K. Kirezieva, G. Hagelaar, J. Rovira, Uyttendaele, M. Jacxsens, et al.
    • Performance assessment of food safety management systems in animal-based food companies in view of their context characteristics: a European study
    • Food Control, 49 (2015), pp. 11–22
    • Luning et al., 2009
    • P.A. Luning, W.J. Marcelis, J. Rovira, M. Van der Spiegel, M. Uyttendaele, L. Jacxsens
    • Systematic assessment of core assurance activities in a company specific food safety management system
    • Trends in Food Science and Technology, 20 (2009), pp. 300–312
    • Martins and Germano, 2011
    • E.A. Martins, P.M.L. Germano
    • Listeria monocytogenes in ready-to-eat, sliced, cooked ham and salami products, marketed in the city of Sao Paulo, Brazil occurrence, quantification, and serotyping
    • Food Control, 22 (2011), pp. 297–302
    • Mataragas et al., 2006
    • M. Mataragas, E.H. Drosinos, P. Siana, P. Skandamis, I. Metaxopoulos
    • Determination of the growth limits and kinetic behavior of Listeria monocytogenes in a sliced cooked cured meat product: validation of the predictive growth model under constant and dynamic temperature storage conditions
    • Journal of Food Protection, 69 (2006), pp. 1312–1321
    • New South Wales Food Safety Authority of Australia, 2008
    • New South Wales Food Safety Authority of Australia
    • Listeria management program
    • NSW Food Authority, Newington, Australia (2008) Available at http://www.foodauthority.nsw.gov.au/_Documents/industry_pdf/listeria-management-program.pdf
    • Podolak et al., 2010
    • R. Podolak, E. Enache, W. Stone, D.G. Black, P.H. Elliott
    • Sources and risk factors for contamination, survival, persistence, and heat resistance ofSalmonella in low-moisture foods
    • Journal of Food Protection, 73 (2010), pp. 1919–1936
    • Sheen and Hwang, 2008
    • S. Sheen, C.A. Hwang
    • Modeling transfer of Listeria monocytogenes from slicer to deli meat during mechanical slicing
    • Foodborne Pathogens and Diseases, 5 (2) (2008), pp. 135–146
    • da Silva do Nascimento et al., 2012
    • M. da Silva do Nascimento, D.M. Brum, P.O. Pena, M.I. Berto, P. Efraim
    • Inactivation of Salmonella during cocoa roasting and chocolate conching
    • International Journal of Food Microbiology, 159 (2012), pp. 225–229
    • da Silva do Nascimento et al., 2010
    • M. da Silva do Nascimento, N. da Silva, I. Francisca da Silva, J. de Cássia da Silva, E. Reolon Marques, A. Regina Barbosa Santos
    • Enteropathogens in cocoa pre-processing
    • Food Control, 21 (2010), pp. 408–411
    • Tamminga et al., 1976
    • S.K. Tamminga, R.R. Beumer, E.H. Kampelmacher, F.M. van Leusden
    • Survival of Salmonella eastbourne and Salmonella typhimurium in chocolate
    • Journal of Hygiene, 76 (1976), pp. 41–47
    • Uyttendaele et al., 1999
    • M. Uyttendaele, P. De Troy, J. Debevere
    • Incidence of Listeria monocytogenes in different types of meat products on the Belgian retail market
    • International Journal of Food Microbiology, 53 (1) (1999), pp. 75–80
    • Zhang et al., 2012
    • L. Zhang, S.R. Moosekian, E.C.D. Todd, E.T. Ryser
    • Growth of Listeria monocytogenes in different retail delicatessen meats during simulated home storage
    • Journal of Food Protection, 75 (2012), pp. 896–905
Source :
Science Direct