Large data volumes are revolutionizing industries, including AI in medical research. This influx of data enables observational studies that harness global statistical evidence for real-world evidence generation. However, conducting such studies can be labor-intensive and prone to inconsistencies in health data standardization due to disconnected communication channels, like repositories, emails, forums, and chats. Moreover, adapting code to different environments during the execution phase can create unscalable and non-reusable analytical frameworks, while the lack of a secure data exchange platform further complicates collaboration. In response, the OHDSI community is developing ARACHNE, a DevOps-enabled research analytics platform for life sciences and an innovative real-world data (RWD) platform for automated orchestration of observational healthcare research across distributed research networks in life sciences, healthcare, academia, and organizations handling patient-level data
Challenge
Use Case Ready For Generative AI
Find out more with SciForce free checklist
As a partner in several projects with OHDSI, our team played an instrumental role in the development of the Arachne Collaboration Network, a platform designed to facilitate the execution of federated data analysis, also known as Real-World Evidence. Sciforce's contributions included shaping the platform's architecture, overseeing both backend and frontend programming, and implementing fundamental DevOps practices.
Solution
We assumed that ARACHNE should automate observational studies and evidence generation through distributed research studies, serving as an automation framework for OHDSI and OMOP CDM studies. Our implementation is divided into two software applications:
ARACHNE Central
Registers and initializes new research projects.
Creates packages with necessary R scripts for each project.
Uses Docker to simplify R distribution installation by wrapping software with all necessary components, ensuring smooth execution regardless of environment. Orchestrating isolated environments across multiple nodes is a challenge we tackled at a different scale in our multitenant virtual datacenter, where Kubernetes automated the full lifecycle of tenant clusters with strict resource and network isolation.
ARACHNE Daemon
Installed at Data Nodes, communicates with ARACHNE Central.
Runs relevant R scripts and executes embedded SQL against the CDM database.
Sends resulting tables back to ARACHNE Central.
Enables distributed execution of OMOP-based observational studies.
Development Journey
Back-end
Language: Java 8
Framework: Spring (used Boot for auto deployment and Security for authorization/access management)
ORM: Hibernate (simplifies data access with object-relational mapping)
Build Tool: Maven (for dependency management and project build)
Front-end
Libraries: React + Redux + Reselect
Styling: SASS (using BEM methodology)
Communication: Websockets (for real-time data transfer)
Bundling: Webpack (for module bundling and asset optimization)
Tools
Containerization: Docker (for R script isolation and ease of deployment). How those two combine into a fully automated deployment pipeline is shown in our Amazon ECS case, where containerized services are provisioned and updated entirely through Infrastructure as Code
Cloud: AWS (for scalable infrastructure)
Project Management: Atlassian tools (Confluence for documentation, JIRA for task tracking, Bitbucket for version control)
Data Storage
Big Data: Apache Cassandra (for storing research results)
Impact
We built the main part of the front-end app and its user interface, along with a file storage system. We also added features for faster communication and organized data-driven healthcare insights and multi-user support. Our work helped create a multi-user healthcare research network that makes enabling clinical research automation at scale more straightforward, transparent, and secure. It connects everyone involved in the research process, from those providing scalable real-world data analytics to the people analyzing it, for a smooth end-to-end study.