Brazilian researchers have developed a computer program that locates swimming pools and rooftop water tanks in aerial photographs with the aid of artificial intelligence to help identify areas vulnerable to infestation by Aedes aegypti, the mosquito that transmits dengue, zika, chikungunya and yellow fever.

The innovation, which can also be used as a public policy tool for dynamic socio-economic mapping of urban areas, resulted from research and development work by professionals at the University of São Paulo (USP), the Federal University of Minas Gerais (UFMG) and the São Paulo State Department of Health's Endemic Control Superintendence (SUCEN), as part of a project supported by FAPESP. An article about it is published in the journal PLOS ONE.

Our work initially consisted of creating a model based on aerial images and computer science to detect water tanks and pools, and to use them as a socio-economic indicator."

Francisco Chiaravalloti Neto, last author of the article

He is a professor in the Epidemiology Department at USP's School of Public Health (FSP), with a first degree in engineering.

As the article notes, previous research had already shown that dengue tends to be most prevalent in deprived urban areas, so that prevention of dengue, zika and other diseases transmitted by the mosquito can be made considerably more effective by use of a relatively dynamic socio-economic mapping model, especially given the long interval between population censuses in Brazil (ten years or more).

"This is one of the first steps in a broader project," Chiaravalloti Neto said. Among other aims, he and his team plan to detect other elements of the images and quantify real infestation rates in specific areas so as to be able to refine and validate the model.

"We want to create a flow chart that can be used in different cities to pinpoint at-risk areas without the need for inspectors to call on houses, buildings and other breeding sites, as this is time-consuming and a waste of the taxpayer's money," he added.

Machine learning

A previous study used artificial intelligence (AI) to detect water tanks and pools in Belo Horizonte, capital of Minas Gerais state. The researchers first presented satellite images of the city to a computer algorithm with tanks and pools already identified. The deep learning program then found patterns in the images that would make detection possible anywhere, and over time acquired the capability of distinguishing tanks and pools in photographs on its own.

"It's genuine machine learning, a sub-area of AI," said Jefersson Alex dos Santos, a professor in the Computer Science Department at UFMG, and founder of its Pattern Recognition and Earth Observation Laboratory (PATREO).

The more recent study focused on Campinas, the third-largest city in São Paulo state by population. Four areas were chosen, each with different socio-economic conditions according to the census. A drone with a high-resolution camera took aerial photographs of the areas, and two datasets were created, one for water tanks and the other for pools.

The next step entailed training the model and transferring the lessons learned. "We trained the model on Belo Horizonte and applied it to Campinas," Santos said. With the images obtained in Campinas, the model became more reliable for the region, achieving accuracy rates of 90.23% and 87.53% for pools and tanks respectively.

Socio-economic indicator

When the algorithm was fully trained, the researchers used other images to detect tanks and pools in the four selected areas of Campinas and cross-referenced them with the census data. The results of the analysis showed larger numbers of roof tanks per square meter in poorer areas and more pools in wealthier areas.

Even these preliminary findings were useful to predict probable breeding grounds for A. aegypti. "It's not the final methodology, but it could serve as a basis for a relatively simple practical application such as developing software to map city districts with a high risk of dengue outbreaks," Santos said.

According to Chiaravalloti Neto, the model can be used for much more than controlling dengue and other mosquito-borne diseases. "The nation updates its socio-economic database about every ten years, with each population census. Our method could be used for more frequent updates, which in turn could be used to combat other diseases and problems," he said, adding that more markers can be found in future studies based on aerial images, so as to refine the algorithms and make them even more accurate.

Drone or satellite imagery?

Although the aerial photographs of Campinas were taken by a drone, the researchers expect the final methodology to use satellite imagery. "We used a drone because it was a pilot project, but large-scale remote sensing and scanning with drones is expensive," Chiaravalloti Neto said.

"Also, drones have relatively little range," Santos added. "For a large-scale project in a major city, we'll need satellite imagery." The Belo Horizonte survey used satellite images successfully. These must be high-resolution images so that the software can recognize patterns. Access to this type of image is, fortunately, becoming easier, he said.

The methodology may seem costly, but actually it saves time and money by avoiding the need for in-person house calls to map potential breeding grounds. Instead, the city's public health workers can use the data obtained remotely and processed by AI to select priority areas for physical inspection more assertively.

Next steps

The model currently cannot detect whether water tanks are properly sealed or whether pools are treated to prevent mosquitoes from laying eggs in them. "The methodology could be refined so as to be capable of distinguishing between properly treated tanks, pools, etc., and others that can or do serve as breeding grounds for the mosquito," Chiaravalloti Neto said. Detection of such patterns and other signs of potential breeding grounds would make the algorithm even more useful to public health departments.

The researchers are now installing traps to catch mosquitoes on some 200 street blocks in Campinas. The state of the properties is being carefully assessed, particularly to predict whether the mosquito is likely to breed there. Socio-economic indicators will also be analyzed. The next step will entail assessment of aerial images of the areas using the logic described above to classify the risk of the presence of A. aegypti and the diseases it transmits.

"As we observe these urban areas, we'll build a model that prioritizes dengue control measures for the entire city, and then for the rest of Brazil," Chiaravalloti Neto said.

In addition to FAPESP, the researchers were funded by Serrapilheira Institute, the National Council for Scientific and Technological Development (CNPq), USP's Office of the Pro-Rector for Research, and FAPEMIG, the Minas Gerais Research Agency. SUCEN provided structural support.