Scaling the Creation of Activities with Artificial Intelligence and Data

By Ali Marques - Data Analyst

Since the emergence of ChatGPT at the end of 2022, artificial intelligence (AI) has played a central role in various industries seeking innovation, including education. However, when it comes to public education, we still lack examples of innovative solutions.

Developing active learning methodologies with ChatGPT

The AprendiZAP is a digital tool whose mission is to be the best and most practical digital learning tool for students at public schools in Brazil. We currently offer more than 2,000 lessons free of charge, but demand for more activities, especially those involving active learning methodologies, was high.

To address this demand quickly, we combined technological innovation with an impact on public education: using AI to make the content creation process scalable , while having it reviewed by specialist teachers in order to maintain our high content standards.

Active learning methodologies have the power to engage students as the central figures in the learning process. Integrating ChatGPT, an advanced natural language processing tool, into the creation of personalized active learning methodologies made it possible to create more than 8,000 activities in just a few days.

Collaboration between teachers and technology

One of the crucial characteristics of our project was the close collaboration between technology and human expertise. Experienced teachers from the public school system not only reviewed and adjusted the activities proposed by ChatGPT, but also contributed valuable insights to shape the content to meet the real needs of classrooms.

Reviewing the materials was made easier by exporting them to Google Docs. But everything comes at a price: By making the teachers’ review process easier, we produced more than 8,000 unstructured files that had to be processed before being added to the platform.

The role of the data team in optimizing the process

Among the responsibilities of anyone working in some capacity in the data field, one of the main skills we develop is data exploration and processing, whether the data is structured or unstructured. For this reason, the AprendiZAP data team was responsible for making the content from the more than 8,000 files available to the development team. This stage is commonly called ETL: Extract (extraction), Transform (transformation), and Load (loading or making available).

The main objective of this data processing was to extract and divide the sections contained in the activity document. It was an iterative process to understand the main section markers—such as titles, for example—which varied among the files.

Example of content for section extraction.

To extract each section, we used regular expressions (or regex), which consist of character patterns used to search for, identify, and manipulate character strings that fit a given format. In this way, it was possible to provide the development team with segmented objects that were easy to manipulate.

Final JSON file, properly segmented.

Monitoring segmentation quality

As mentioned, the files differed in the terms used to designate the beginning of a section, while other documents did not follow the same sequential order of sections. So how could we determine whether the segmentation performed was correct across more than 8,000 pieces of content? It is easy to identify whether segmentation is correct in a sample, but with thousands of pieces of content, the situation is different.

We implemented an approach based on counting the characters in the sections of each piece of content, obtaining a normal distribution for each type of section. This made it possible to identify outliers, that is, points outside the norm. For example, when a section contained more than 1,000 characters in an item, while the average number of characters in an item was 50, something was wrong.

Distribution of characters by items related to a section.

It was during this process that we identified variations in titles, special characters, and other discrepancies, which could then be fed back into the specifications for the regular expressions. In the end, only a few materials actually required human intervention and correction. Even so, this was work reduced for our educational team, which had to deal with less than 2% of the generated content.

Technology and public education: A near-term horizon

Our project at AprendiZAP is a powerful demonstration of how technology can be applied to transform public education. Collaboration between technology and human experts generated a scalable, high-quality model. In addition, it was an example of how data professionals can apply the knowledge they acquire in their day-to-day work. It is the first step (of many) in using artificial intelligence to create social impact in Brazilian public education.