Skip to content

Regression and Classification Machine Learning Model Management Software Tool for Energy Toolbase

Project Category: Software

Come see us at the Capstone Design Fair

Capstone Design Fair is scheduled for April 5th, 2022 from 9:30AM to 1:00PM (MST)

About our project

Background

Energy Toolbase (ETB) develops a suite of products for solar power and battery management. An offered feature is the ability for batteries to automatically store and release energy as needed. To predict when to release and when to store, ETB’s data scientists develop machine learning models for forecasting future energy usage and weather conditions.

ETB’s data science team explores a variety of model versions and techniques, thus organizing and comparing models are necessary steps in selecting the best models. These steps are currently done manually, which is slow and inefficient. This situation will also only worsen over time as more models are developed.

Thus, ETB required a solution to both organize and compare their models.

Meet our team members

Jake Liu

Software Engineering ’22

  • Project Manager
  • Full-Stack Developer
  • Tech Lead

After interning at ETB as a software developer for 16 months, Jake served as the primary liaison between the team and ETB. Jake helped to drive the project through development by managing its direction and using his programming skills to complete some of its most crucial and intricate objectives.

Jake’s interests lie in software design, problem-solving, and machine-learning methods.

Jannalie Taylor

Software Engineering ’22

  • Full-Stack Developer
  • Lead UI Developer

Jannalie completed a 16 month internship and was the lead designer and software developer for both the UI and the classification models. Jannalie’s expertise and ability to debug programs allowed her to fix dozens of bugs to improve the UI experience. Jannalie is easy to work with and pushes projects forward.

Along with furthering her programming skills, Jannalie bakes and sews in her free time.

Jean-David (JD) Rousseau

Software Engineering ’22

  • Full-Stack Developer

JD worked at General Dynamics for 16 months as a firmware/software developer.

JD’s programming ability allowed him to work on numerous issues spanning both projects, resolved numerous bug, and improved the final product through refactors and enhancements. JD’s attention to detail also allowed him to find dozens of bugs in the system for the team to fix before delivery.

Outside of school, JD furthers his programming with personal projects and is an avid martial artist.

Raphael (Matan) Castillo

Software Engineering ’22

  • Full-Stack Developer
  • Database Developer

Matan worked at General Dynamics for 16 months as a firmware/software developer.

Matan was a principal developer for the model registry package and the database interactions. Matan easily completes assigned tasks on time.

From a young age, Matan has been interested in music. He is currently a level 8 in piano with the Royal Conservatory of Music.

Nicholas Cervania

Software Engineering ’22

  • Full-Stack Developer

Nicholas worked as a Junior Software Developer for four months at OptEM Engineering and eight months at the Canadian Space Agency.

Nicholas’ ability to learn new languages and frameworks allowed him to effectively contribute to the project.

In his free time, Nicholas enjoys writing and playing guitar, and is interested in the information security and software development fields of software engineering.

Details about our design

HOW OUR DESIGN ADDRESSES PRACTICAL ISSUES

Our design addresses ETB’s issues of model management and comparison using a two-pronged approach. First, a model library for storing ETB’s models was created with an interface to integrate it with existing software systems created by ETB’s data science team. Second, an interactive web application for managing models was created, allowing ETB’s data scientists to easily upload, search, edit and delete models. Additionally, collections of graphs can be created and saved, simultaneously comparing the performance of several different models. Furthermore, several machine learning models were developed to demonstrate the functionality of the system and investigate different types of models.

WHAT MAKES OUR DESIGN INNOVATIVE

Our design is innovative in that it added two entirely new software tools to ETB’s toolkit for model comparison and storage. The first tool acts as an API into a database which tracks models. This provides the majority of the model management functionality. The second tool is a UI which allows users to easily interact with the database and to compare models.

Our design is also innovative in the fact it approached (with the counsel of ETB’s data science team) energy usage as a function of weather as a classification problem rather than a regression problem. Regression machine learning models try to output a value such as 5, meanwhile classification machine learning models try to find a bucket in which this value would fit, such as between 3 and 7. The vast majority of literature approaches this problem as a regression problem as well. The classification method would be used to show the confidence about a prediction, allowing ETB to choose safer energy strategies over optimal strategies.

WHAT MAKES OUR DESIGN SOLUTION EFFECTIVE

Our solution is effective as it addresses the issues faced by ETB’s data science team.

It standardizes and centralizes models that ETB creates. It allows members of ETB’s data science team to upload models into a shared environment with versioning and additional metadata to keep track of models. As well as the ability to search for these models and then download them for use.

The solution also allows for quick comparison of models by creating shareable dashboards that any user can see. This comparison tool allows ETB to make decisions about the based on various performance metrics. Such as whether a model is better than another or even whether certain classes of models perform better than another class of models.

HOW WE VALIDATED OUR DESIGN SOLUTION

Our team validated our solution using a multi-faceted approach.

  • Our team created our own models, which helped with the classification model investigation, as well as using them to validate various parts of the system. Pushing these models through the system quickly unearthed any issues our system had, and helped prove our system worked with real data.
  • Our team also created 200 unit tests to ensure functionality and code quality remains constant through project iterations.
  • Our team enlisted the help of ETB’s data science team to use our system and provide feedback for the functionality and ease of use of the system.
  • We also measured key performance metrics that were outlined in the design of the project. These measured values were compared to these outlined values to determine the validity of our solution.
FEASIBILITY OF OUR DESIGN SOLUTION

Our key focus in this project was to attain a usable system for ETB. This focus was accomplished through careful planning and prioritizing of the different aspects of the project. We used a JIRA board to manage and prioritize our work amongst the many project components. We consulted with our academic advisor and sponsor representatives to discuss the possibility of certain features. We constantly triaged the bugs and features and included ETB on discussions for directions in the project. For example the classification model investigation was of low priority compared to the model registry and comparison features.

Partners and mentors

We want to thank the many people who helped us with this project.

  • Our academic advisor Dr. Roberto Medeiros de Souza, who guided us and provided tremendous help with the Machine Learning aspect of the project. Thank you again for patiently and diligently answering all our questions.
  • The sponsor ETB, in particular Prudhvi Tella and Jielian Guo, helped us by guiding our process and providing valuable feedback to all our questions and design considerations.

Our photo gallery