Friend´s Autolarm
| Reference number | |
| Coordinator | Stiftelsen Friends - Elever Mot Mobbning - Stiftelsen Friends |
| Funding from Vinnova | SEK 600 000 |
| Project duration | November 2022 - November 2023 |
| Status | Completed |
| Venture | Learning and meeting places |
| Call | Start your AI journey: For organizational learning and practical use of artificial intelligence in municipalities and civil society spring 2022 |
Purpose and goal
The project aimed to use AI to analyze surveys from school students in Sweden and detect early signs of bullying. Data was collected from 30,000 surveys, with 600 annotated responses in categories of victimized/not victimized. The model´s goal was 85% accuracy, 80% precision and recall, and an F1-score of 0.82. The model was designed to be sensitive and rather produce false positives. Pre-built language models like ChatGPT and Llama2 were compared with the project´s AI to measure effectiveness.
Expected effects and result
We utilized our own survey tool tailored for schools and developed AI to detect abuse and bullying. When compared against an LLM, the latter outperformed the former. Our AI discerned just ´bullying´ or ´no bullying´, while the LLM differentiated several categories, offering deeper insights. This result, driven by our model´s data limitations, was foreseeable. LLMs might outshine in precision. However, with ample data, the custom model built for the project holds potential.
Planned approach and implementation
The project used deep learning and LSTM for sentiment analysis of data from Friends and app reviews. Data processing included text cleaning and the creation of numerical representations. Data was trained in different rounds, and after each training round, performance metrics were evaluated. Adjustments were made to improve the model´s performance. Pre-trained language models were tested and showed equivalent performance. Collaboration with Friends ensured the quality of the text analysis