ComplyShield Monitoring Screen

From project inception, pragmatism and practicality were one of the main goals of our project. The development of production-level software to be deployed to solve real-life challenges was a foundational pillar of our product.

Currently our client, Vantedge Oil & Gas (O&G), has to include thousands of government legislations, codes, and standards as part of their ComplyShield platform. These government documents are subject to change over-time and it is Vantedge’s responsibility to maintain the most up-to-date versions. Prior to our solution, this was a manual process where our client periodically surveyed relevant government sites and appended any updates in an excel sheet. This process was extremely time consuming and prone to human error during data entry. Furthermore, the reliance of a shared excel sheet is problematic as there is no robust way to track changes, excel does not scale well as the dataset grows, and it is not secure.

Our design will reduce the resources needed to maintain the government data on ComplyShield, remove the possibility of human error, and provide a scalable and secure way of managing updates.

To address the issues faced by Vantedge with their current solution, our monitoring screen solution automates the entire process of legislation updates collection. We built a multitude of services that run everyday and check government sites for new updates, fetch the relevant data, and store it in a database.

To create a seamless experience for the end users, we developed two new components within the ComplyShield application. The first is the manual review queue where analysts can review, edit, and finalize parsed legislative updates to ensure their correctness. Once the updates are finalized the client can publish them to the second component: the monitoring screen. This is where the updates will be displayed in full detail, so that the client can process them and integrate them into their system to serve their clients.

The versatility combined with a very specific scope makes our design solution incredibly effective. This first-of-its-kind solution parsed over 3000 Government of Canada legislation entries upon initial deployment, spanning from Canadian Gazette to Parliament of Canada data. Minimal supervision of this continuous system is required, but also contrarily permits precise fine-grained control of the types of data being filtered through. The manual review queue system enables Vantedge data analysts to assess each individual legislation entry in remarkable detail. Our solution runs daily during non-peak hours of the day which mitigates concentrated loads on company servers. Each entry in the database can be easily reviewed and edited, providing analysts effective control over their own data.

Future expansion of the scope to provincial-level and American legislation data will warrant many more thousands of legislation entries being processed and reviewed. This will provide Vantedge the unique opportunity of furthering their industry-leading software all across North America.

User experience and future maintainability were both paramount priorities throughout our design and development process. By leveraging bi-weekly sprints and thoughtfully working with our sponsors to solicit feedback, we consistently iterated on our product to satisfy the business requirements. We prioritized production-quality code to ensure future scalability and maintainability. We tested our programmed solution by writing unit tests for 14 modularized services and through manual testing of the user experience performed by Vantedge analysts.

Our measure of success was to have a fully passing test suite which covered our entire programmed solution. The outputs of the unit tests followed our sponsor’s specifications, with the condition that all 89 new and 1677 pre-existing tests must pass before merging our work with the existing production solution. An example of a unit test we wrote was to check within the logger service if an error in the parser (due to a missing section in the HTML page) was recorded correctly. Lastly, we also thoroughly tested our helper services to ensure each component of our backend was working as expected.

The project was programmed following the application’s existing code guidelines. This involved following SOLID principles, using proper formatting and linting configurations, and to program the solution with scalability in mind due to the constant influx of legislation updates. This means our monitoring screen solution is set up well for production use as it is easily maintainable and runs periodically without human intervention to collect legislation data.

We also created services within the solution such as a logger and emailing service which keeps the stakeholders of the project up to date with its status. Each run is logged with proper outputs and error messages and every week, a log is outputted to stakeholders providing crucial information about the services. The detailed log information allows Arcurve engineers to easily patch any unintended behaviour happening within the application.