Skip to main content

Effects of four soil-originated Bacillus spp. on the great spruce bark beetle, Dendroctonus micans (Kugelann) (Coleoptera: Curculionidae, Scolytinae)

Abstract

Fifty bacterial cultures were isolated from 156 soil samples. In order to identify six bacteria in both systems, a fatty acid methyl ester (FAMEs) analysis was conducted, and carbon utilization profiles were assessed, using microbial identification, Biolog Microplac Systems, and the VITEK bacterial identification systems (bioMerieux, Prod. No. 21341 and 21342). Results showed that four species of Bacillus ssp., isolated from soil, were safe and efficient biological control agent for plant pests in Ordu, Turkey. These bacteria were Bacillus mycoides, B. cereus, B. thuringiensis, Paenibacillus validus, B. atrophaeus, and Arthrobacter globiformis. Laboratory tests were conducted to assess the potential of the isolates against the great spruce bark beetle, Dendroctonus micans (Kugelann) (Coleoptera: Curculionidae, Scolytinae) (Curculionidae, Scolytinae). Mortality rates of larvae and adults were 60 and 50%, 40 and 30%, and 80 and 70% when using B. mycoides, B. cereus, and B. thuringiensis, respectively. P. validus, B. atrophaeus, and A. globiformis showed insufficient rates of mortality, 30 and 20% on the larvae and adults of D. micans, respectively. Also, these isolates had no antimicrobial effect on pathogen microorganisms. They have, however, a lethal effect on some insect groups that are agriculture and forest pests. The results indicated that the Bacillus isolates in question can be used as one of the biological control agent.

Background

Microbial pest control agents (MPCAs), especially the products of different Bacillus thuringiensis (Bt) subspecies, are utilized more and more in pest management programs to combat against the larvae of certain insect vectors of agricultural crops and nuisance pests of humans (Miller 1990). An eco-friendly substitute to pesticides is provided by biological control agents to treat plant diseases. However, producers persist in utilizing chemical control rather than the alternative provided by biological control. The ignorance on this matter generally leads to the collapse of a biocontrol agent (Ardakani et al. 2010). Biocontrol can be achieved as long as the biological environment where the agent will be used is well known and the know-how of the production of a reliable and resistant formulation is available (Emmert and Handelsman 1999). Some Gram-positive bacteria are superior to their Gram-negative counterparts in terms of their natural formulation, the spore (Charles et al. 2008). Bt and its close relatives are a facultative anaerobic, Gram-positive bacterium comprising idiosyncratic protein inclusions contiguous to the endospore. The parasporal inclusions are composed of various insecticidal crystal proteins (ICP) (Van Frankenhuyzen 2009), which determine the shape of the crystals (Otieno 2010).

As its range extended from east to west progressively, the great European spruce bark beetle Dendroctonus micans (Kugelann) (Coleoptera: Curculionidae, Scolytinae) headed from Georgia to Turkey. The first report on its presence in Turkey came in 1966 (Yaman and Radek 2008). In spite of the comprehensive and costly endeavors to control the beetle, it continues to damage the economy of Turkey. In more than three decades, approximately 250.000 ha have adversely been impacted and no fewer than ten million spruce trees have been killed (Yaman and Radek 2008). D. micans is a notable defoliator of species of pine trees in Turkey and in the rest of the Mediterranean (Yaman and Radek 2008). It is also found in other countries, which have spent a great deal of money for controlling it (Grégoire 1984; Yaman et al. 2010).

It has been reported that Bt subspecies are quite unique in terms of their potential for Coleoptera, Diptera, and Lepidoptera, where they are not directly toxic to non-target arthropods (Miller 1990). Giving rise to little tree mortality, D. micans is typically observed at low levels within its natural range. Still, outbreaks which lead to extensive tree mortality are observed every now and then. The majority of the outbreaks take place along the front edge of the geographic range of D. micans and not within the inner part of the range. Trees are killed on account of the girdling action of larval feeding, which may occur during a few years. D. micans enlarged its range into Europe (France and the UK) and into southwestern Asia in the late 1900s, and therefore, the outbreaks took place on no less than 200,000 ha of spruce forests (Vakula et al. 2016).

A pilot study on pine trees was performed to analyze the body pathogens and to determine the most aggressive one on the holes in the pine tree body of young and old plants.

Materials and methods

Insect collection

The larvae and adults of the D. micans pine trees beetle were obtained from the frost fields near Ordu, Turkey. Cultures were maintained in a laboratory set up by the Turkish Ministry of Forestry in Ordu.

Soil sampling

The number of soil samples collected from 18 districts of Ordu from 2009 to 2010 was 156. The collected samples were investigated between 2012 and 2015. Site location, sampling date, elevation, and associated vegetation were recorded. In all instances, each sample of soil that amounts to 10 g of surface soil specimen scraped within 2 to 5 cm depth with a sterile spatula. The specimens were placed in sterile bags and kept at 4 °C. The specimens were extracted with a hand shovel, put in polyethylene bags to prevent water loss, and kept in coolers during their transfer to the laboratory. The soil sample was suspended in sterile physiological saline (1:5, w/v), mixed arduously, and allowed to stand with no disturbance. A 1.5-ml clear supernatant was heated at 80 °C for 10 min in a water bath to remove the no spore-forming organisms (Yaman et al. 2002). The heat-treated specimens of 0.1 ml was spread on nutrient agar plates, incubated at 28 °C for 48 to 96 h (Lee et al. 1995), and analyzed to pick bacterial colonies. They were disinfected by subculture on plates, and the cultures were identified by a variety of tests. Finally, four Bacillus species and two other species were extracted from the soil.

Identification of bacterial isolates

Fatty acid profiles of isolates were determined according to the methods proposed by Sasser (1990). For this process, a model 5898A microbial identification system (Hewlett-Packard, Palo Alto, CA) and TSBA (Tripticase Soy Broth Agar) database in the microbial recognition system software (MIDI; Microbial ID, Inc., Newark, DE) was used. A few isolated bacterial strains were detected, using fatty acid profiles designated utilizing the Microbial Identification System with TSBA, and carbon substrate utilization footprints were assessed using the Biology GN and GP database with Microlog software in Biolog Microplac system (Biolog Inc., Hayward, CA) at the Department of Plant Protection in Erzurum. The isolates were kept at the Department of Plant Protection, Faculty of Agriculture, Atatürk University, and the Department of Biology, Faculty of Science, Ordu University, Turkey. Other isolated bacterial isolates were initially stained using Gram stain for Gram-positive or Gram-negative identification, and experiments were conducted to determine several biochemical reactions. Afterwards, isolated bacteria were identified, using VITEK bacterial identification systems. Bacillus species were also stained to detect crystal protein.

Extracellular enzyme production capabilities of Bacillus isolates

The quality of the production of amylase, chitinase, xylanase, lipase/esterase, pectinase, protease, and cellulase extracellular enzyme in Bacillus isolates was investigated, using the following media and techniques:

  • Amylase extracellular enzyme: Starch was laid to incubate overnight for one night in order to observe the amylase activity. The tests yielded some positive results in the zones observed by staining with iodine solution following reproduction (Çoşkun 2010).

  • Kitinase extracellular enzyme: Colloidal chitin-containing nutrient was used (Roberts and Selitrennikoff 1988).

  • Xylanase extracellular enzyme: Xylan-containing nutrient was used to detect xylanase activity in Bacillus. The transparent zones around the colony were evaluated as a positive result (Roy and Rowshanul 2009).

  • Lipase/esterase extracellular enzyme: Lipase/esterase was used for the investigation of the presence of extracellular enzyme. Agar containing tributyrin, Tween 20 and Tween 80 toothpicks were used. Cultivation was performed for 3 days at 30 °C (Kugimiya et al. 1980). A 2.5% olive oil was added to Rhodamine B in agar bread to verify lipase activity in Bacillus (Haba et al. 2000; Litthauer et al. 2002).

  • Pectinase extracellular enzyme: To investigate the presence of pectinase, two different media were used. The test was conducted at room temperature for 10 min. The open zone formation around the colonies was evaluated as a pectinase producer (Altan 2004).

  • Protease extracellular enzyme: Protease medium was used to observe protease activity in the Bacillus isolates. The resulting zones were interpreted as positive for protease activity.

  • Cellulase extracellular enzyme: CMC medium was used to investigate the cellulase activity in Bacillus isolates (Sambrook and Russell 2001). After growth, enzyme activity was observed (Çoşkun 2010).

Experimental infections with D. micans larvae and adults

Experimental infections were performed for each isolate to evaluate its potential on larvae and adults of D. micans, which fed under the barks of spruce trees and cause damage. For that reason, the larvae and adults were fed with barks dipped into the suspended bacterial cells incubated at 28 to 37 °C for 24 to 72 h (Ertürk et al. 2008). The control group was fed with barks dipped with sterilized water. The specimens were reared in groups of 30 insects in containers to allow air flow. To that end, spruce barks with 4 cm in diameter were dipped in a suspension of bacterial cells and then put into containers (8.5 × 6.5 cm). The specimens were put on diet in containers for each assay. After 48 h, the insects were provided by fresh untreated barks every 24 h (Guire et al. 1997). The control group was provided by barks dipped in sterile water for the first 48 h and then fresh untreated bark every 24 h. All groups were stored at 26 °C and 60% RH (Mitchell and Smith 1985). Observations were conducted on a daily basis, and dead larvae or adults were removed right away. All these tests were conducted in triplicate on different days. Abbott’s formula (Abbott 1925) was employed to set the data right (Reed and Halliday 2001).

Results and discussion

Four Bacillus species and two other species were isolated from the soil and identified as Bacillus mycoides, B. cereus, B. thuringiensis, Paenibacillus validus, B. atrophaeus, and Arthrobacter globiformis. Bacterial isolates were identified using fatty acid profiles, bon substrate utilization footprints, and then VITEK. The ability of Bacillus isolates to produce amylase, lipase/esterase, chitinase, xylanase, pectinase, protease, and cellulase extracellular enzyme was qualitatively investigated. The results of the enzyme activity outside the cell of the isolates of Bacillus isolates are given in Table 1. Effects of all isolated bacteria were tested on the adults and larvae of D. micans. Laboratory experiments conducted to demonstrate potential of these isolates indicated that they all had, more or less, an insecticidal impact on the larvae and adults of the bark beetle. B. mycoides had mortality rates of 60 and 50% on the larvae and adults of D. micans, respectively. A novel strain of B. mycoides strain AQ 726 that generates a metabolite pesticidal activity was observed by Heins et al. (1999) who developed a technique to restore or preserve a plant from such insect infestations as corn rootworm. The technique involved the application of a fair quantity of novel metabolite-producing bacterial strain to the plant or its environment. This bacterial strain was a supernatant comprising a metabolite acquired from a whole broth culture of the strain or the metabolite itself. B. cereus had 40 and 30% mortality on the larvae and adults, respectively. B. cereus was also accepted as an insect pathogen. It was observed that B. cereus secreted proteins that immobilize and kill cockroaches, Blattela germanica, upon injection (Man et al. 2013). The highest effect was determined with B. thuringiensis on the larvae and adults of the bark beetle. This bacterium showed 80 to 70% mortality rate on the larvae and adults of D. micans, respectively. The findings of this study showed that the larvae were more vulnerable to the isolated bacteria than the adults (Fig. 1). In the literature, few bacteria other than B. thuringiensis Berliner are lethal to the Colorado potato beetle, Leptinotarsa decemlineata (Say), a major pest of potatoes and eggplant (Martin et al. 2004). In this study, four lethal Bacillus species were found against the bark beetle, while some of which were lethal to other insects as well.

Table 1 The morphological characteristics of bacterial and extracellular enzymes belonging bacteria isolates from soil
Fig. 1
figure 1

The effects of the isolated bacteria on Dendroctonus micans larvae and adults

No studies addressing entomopathogenic bacteria in bark beetles have been found in the literature (Yaman et al. 2010). Wegensteiner (2004) reviewed the bacteria of bark beetles and provided several species recorded by different scientists. A few of these studies are on D. micans. Imnadze (1978) pointed out the presence of B. thuringiensis in D. micans. Yılmaz et al. (2007) isolated seven bacteria from this pest and compared the isolates with the bacteria recorded from bark beetles, where it was found that only two of them had been isolated from those insects before. P. validus, B. atrophaeus, and A. globiformis exhibited the same effect with the mortality rates of 30 and 20% on the larvae and adults of D. micans, respectively. Being a genus of facultative anaerobic, endospore-forming bacteria, Paenibacillus was initially categorized as belonging to the genus Bacillus, but recategorized as a different genus in 1993 (Zhao et al. 2012). The bacteria of this genus are commonly observed in various media (Simonová et al. 2015). Paenibacillus polymyxa (Bacillus polymyxa) is a Gram-positive bacterium, which is also able to fix nitrogen (Vilinska et al. 2008). Being Gram-positive, B. atrophaeus is a biological indicator for sterilization (Christensen and Kristensen 1979). Luo et al. (2012) reported that Bacillus atrophaeus CPB072 had poisoning and controlling effects on potato beetles. It can grow and massively produce spores in a Luria-Bertani culturing medium, at 30 °C and pH 7.0. Inoculation concentration > 108/ml was the optimum condition for fungal infection pathogenicity.

Pesson et al. (1955) reported that Aerobacter scolyti and Escherichia klebsiellaeformis led to adequate mortality rates in Scolytus multistriatus within 72 h. The number of studies on the bacterial pathogens of D. micans is scant. Yılmaz et al. (2007) isolated seven bacteria on D. micans; however, they failed to test their effects on the pest. Conducting experimental infections of isolated organisms from insects is crucial to substantiate the potential of their use as biological control agents.

The findings of the experimental infections are encouraging. However, they are limited to laboratory scale. It is also necessary to conduct field treatments on these isolates. We had some difficulties for keeping the larvae and adults of D. micans in test boxes for a long-term trials. For this reason, probably, Yılmaz et al. (2007) failed to test the effects of these isolates on D. micans. Tonka et al. (2008) accomplished to analyze the laboratory management of entomopoxvirus in Ips typograhus and found that 595 out of 1142 offered beetles were infected by the entomopoxvirus. They tested this virus in a laboratory for several weeks.

Conclusions

Approximately 50 isolates of bacteria were isolated from 156 soil samples. Identification was carried out, and six bacterial species were tested in the present study. FAME analysis and VITEK results are too long to include them in this study. It was demonstrated a number of bacterial species that are lethal to D. micans. Most of the biological control studies on this pest have concentrated on the predatory beetles but also reported a significance potential effect of these bacterial species on the larvae and adults of D. micans. In conclusion, these bacteria are efficient biological control agents to combat D. micans.

References

  • Abbott WS (1925) A method of computing the effectiveness of an insecticide. J Econ Entomol 18(2):265–267

    Article  CAS  Google Scholar 

  • Altan A (2004) Isolation and molecular characterization of extracellular lipase and pectinase producing bacteria from olive oil mills, Master thesis. İzmir Institute of Technology, Institute of Science, İzmir

    Google Scholar 

  • Ardakani S, Heydari S, Khorasani A, Arjmandi R (2010) Development of new bioformulations of Pseudomonas fluorescens and evaluation of these products against damping-off of cotton seedlings. J Plant Pathol 92(1):83–88

    Google Scholar 

  • Charles PE, Ladoire S, Aho S, Quenot JP, Doise JM, Prin S, Blettery B (2008) Serum procalcitonin elevation in critically ill patients at the onset of bacteremia caused by either Gram negative or Gram positive bacteria. BMC Infect Dis 8(1):38

    Article  PubMed  PubMed Central  Google Scholar 

  • Christensen EA, Kristensen H (1979) Biological indicators for the control of ethylene oxide sterilization. Acta Pathol Microbiol Scand B 87B(3):147–154

    PubMed  CAS  Google Scholar 

  • Çoşkun A (2010) Industrial enzymes producing new Bacillus sp. Isolation of strains and characterization, M. Sc. Thesis. Cukurova University, Institute of Science, Adana

    Google Scholar 

  • Emmert EA, Handelsman J (1999) Biocontrol of plant disease: a (Gram-) positive perspective. FEMS Microbiol Lett 171(1):1–9

    Article  PubMed  CAS  Google Scholar 

  • Ertürk Ö, Yaman M, Aslan İ (2008) Effects of four soil-originated Bacillus spp. on the Colorado potato beetle, Leptinotarsa decemlineata (Say). Entomol Res 38:135–138

    Article  Google Scholar 

  • Grégoire JC (1984) Dendroctonus micans in Belgium: the situation today. In: Grégoire J-C, Pasteels JM (eds) Biological control of bark beetles (Dendroctonus micans). Commission of the European Communities, Brussels, pp 48–62

    Google Scholar 

  • Guire MR, Galan-Wong LJ, Tame-Guerra P (1997) Bioassay of Bacillus thuringiensis against lepidopteran larvae. In: Lacey LA (ed) Manual of techniques in insect pathology, pp 91–99

    Google Scholar 

  • Haba E, Bresko O, Ferrer C, Marqués A, Busquets M, Manresa A (2000) Isolation of lipase secreting bacteria by deploying selective substrate. Enzym Microb Technol 26:40–44

    Article  CAS  Google Scholar 

  • Heins SD, Manker DC, Jimenez DR, Marrone PG (1999) U.S. patent no. 5,906,818. U.S. Patent and Trademark Office, Washington, DC

    Google Scholar 

  • Imnadze TS (1978) Characteristics of strains of Bacillus thuringiensis serotype I isolated from bark beetles in Georgia. Soobshch Akad Nauk Gruz SSR 92:457–460 In: Rev Appl Entomol Ser A 68(2616):328

    Google Scholar 

  • Kugimiya W, Otani Y, Hashimoto Y, Tagagi Y (1980) Molecular cloning and nucleotide sequence of lipase gene. Biochem Biophys Res Commun 14(1):185–190

    Article  Google Scholar 

  • Lee HH, Jung JD, Yoon MS, Lee KK, Lecadet MM, Charles JF, Dumanoir VC, Franchon E, Shim JC (1995) Distribution of Bacillus thuringiensis in Korea. Bacillus thuringiensis Biotechnol Environ Benefits 1:201–215

    Google Scholar 

  • Litthauer D, Ginster A, Skein EVE (2002) Pseudomonas luteola lipase: a new member of the 320-residue Pseudomonas lipase family. Enzym Microb Technol 30(2):209–215

    Article  CAS  Google Scholar 

  • Luo HD, Yan JL, Yu Y, Tan WZ (2012) Isolation, screening and identification of bacterial agents for biological control of Colorado potato beetle. Chin Agric Sci 45(18):3744-3754

  • Man V.F.P., Killeen Y.M., Denoma M.C., Herrera K.J., Barcay S.J., Pattison W.J. 2013. U.S. Patent Application No. 13/898,035.

    Google Scholar 

  • Martin RE, MacIntosh BJ, Smith RC, Barr AM, Stevens TK, Gati JS, Menon RS (2004) Cerebral areas processing swallowing and tongue movement are overlapping but distinct: functional magnetic resonance imaging study. J Neurophysiol 92:2428–2443

    Article  PubMed  Google Scholar 

  • Miller JC (1990) Effects of a microbial insecticide, Bacillus thuringiensis kurstaki, on nontarget Lepidoptera in a spruce budworm-infested forest. J Res Lepidoptera 29(4):267–276

    Google Scholar 

  • Mitchell LS, Smith RV (1985) Recreational geography: inventory ans prospect. Prof Geogr 37(1):6–14

    Article  Google Scholar 

  • Otieno PK (2010) Isolation, characterization and evaluation of potency of native Bacillus thuringiensis against maize insect pests and aflatoxin producing fungi, Doctoral dissertation. University of Nairobi Kenya

  • Pesson P, Toumonoff C, Chararas C (1955) Etude des epizooties bacteriennes observees dans les elevages d’insectes xylophages. Ann Epiphyties 6:315–328

    Google Scholar 

  • Reed JP, Halliday WR (2001) Establishment of Cry9C susceptibility baselines for European corn borer and southwestern corn borer (Lepidoptera: Crambidae). J Econ Entomol 94(2):397–402

    Article  PubMed  CAS  Google Scholar 

  • Roberts WK, Selitrennikoff CP (1988) Plant and bacterial chitinases differ in antifungal activity. J Gen Microbiol 134:169–176

    CAS  Google Scholar 

  • Roy N, Rowshanul HM (2009) Isolation and characterization of xylanase producing strain of Bacillus cereus from soil. Iran J Microbiol 1(2):49–53

    Google Scholar 

  • Sambrook J, Russell DW (2001) Molecular cloning: a laboratory manual 3. Press. Cold Spring Harbor Laboratory Press, New York

    Google Scholar 

  • Sasser MS (1990) Identification of by gas chromatography of cellular fatty acids, Technical note 101. Microbial ID, Inc, Newark

    Google Scholar 

  • Simonová MP, Kme V, Lauková A (2015) Occurrence and characterization of Paenibacillus sp. isolated from rabbits. Folia Microbiol 60(2):97

    Article  CAS  Google Scholar 

  • Tonka T, Weiser J, Pultar O (2008) Entomopoxvirus in the spruce bark beetle, Ips typograhus and its laboratory management, IUFRO WG 7.03.10 Methodology of Forest Insect and Disease Survey in Central Europe, September 15 to 19. Strbske Pleso, Slovakia

    Google Scholar 

  • Vakula J., Zubrik M., Galko J., Gubka A., Kunca A., Pavlík J., Nikolov C. 2016. On non-native spruces in Slovakia–short communication

    Google Scholar 

  • Van Frankenhuyzen K (2009) Insecticidal activity of Bacillus thuringiensis crystal proteins. J Invertebr Pathol 101(1):1–16

    Article  PubMed  CAS  Google Scholar 

  • Vilinska A, Rao KH, Forssberg E (2008) Microorganisms in flotation and flocculation of minerals: an overview. In: International Mineral Processing Congress. American Science Press Inc., USA pp 22–39

  • Wegensteiner R (2004) Pathogens in bark beetles. In: Lieutier F, Day KR, Battisti A, Gregoire JC, Evans HF (eds) Bark and wood boring insects in living trees in Europe, asynthesis. Kluwer, Dordrecht, pp 291–313

    Chapter  Google Scholar 

  • Yaman M, Ertürk Ö, Aslan I (2010) Isolation of some pathogenic bacteria from the great spruce bark beetle, Dendroctonus micans and its specific predator, Rhizophagus grandis. Folia Microbiol 55(1):35–38

    Article  CAS  Google Scholar 

  • Yaman M, Nalçacioğlu R, Demirbağ Z (2002) Studies on bacterial flora in the population of fall webworm, Hyphantria cunea Drury. (Lepidoptera:Arctiidae). J Appl Entomol 126:470–474

    Article  Google Scholar 

  • Yaman M, Radek R (2008) Pathogens and parasites of adults of the great spruce bark beetle, Dendroctonus micans (Kugelann)(Coleoptera: Curculionidae, Scolytinae) from Turkey. J Pest Sci 81(2):91–97

    Article  Google Scholar 

  • Yılmaz H, Sezen K, Katı H, Demirbağ Z (2007) The first study on the bacterial flora of the european spruce bark beetle, Dendroctonus micans (Coleoptera: Scolytidae). Biologia 61(6):679–686

    Google Scholar 

  • Zhao J, Wu X, Nie C, Wu T, Dai W, Liu H, Yang R (2012) Analysis of unculturable bacterial communities in tea orchard soils based on nested PCR-DGGE. World J Microbiol Biotechnol 28(5):1967–1979

    Article  PubMed  CAS  Google Scholar 

Download references

Availability of data and materials

Data and materials in this study can be used as reference by other researcher.

Author information

Authors and Affiliations

Authors

Contributions

Majority contribution belongs to the first author. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Duygu Odabas Alver.

Ethics declarations

Ethics approval

Not applicable. Ethical approval is not required for this study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Odabas Alver, D., Ertürk, Ö. Effects of four soil-originated Bacillus spp. on the great spruce bark beetle, Dendroctonus micans (Kugelann) (Coleoptera: Curculionidae, Scolytinae). Egypt J Biol Pest Control 28, 70 (2018). https://doi.org/10.1186/s41938-018-0074-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41938-018-0074-8

Keywords