The SoM Research Computing Center provides research high-performance computing (HPC) and data management support services, and can advise researchers on the most effective means of allocating resources, managing data, or transferring data to optimize or reduce costs.
- Training outside of initial onboarding
- Data migrations to/from cluster(s)
In addition, we offer Software Engineering and HPC Consultation services.
- Find solutions to confusing errors and explain the underlying causes.
- Ensure proper resource allocations to get jobs running sooner
- Parallelize jobs to maximize throughput with control to avoid over utilizing cluster resources
- Work with lab staff to create portable and reproducible workflows that maximize job scheduling. In one case, an 8-hour job with many tedious manual steps was reduced to a fully automated workflow that ran in 30 minutes.
- Evaluate jobs that run slow or longer than expected to find bottlenecks.
- Create software environments that are compatible with the cluster, are portable, and reproducible.
- Help users make use of containers that provide reproducible software that can support software that wouldn't otherwise be available on a HPC cluster.
- Package ML models in cluster-compatible containers
Create jobs or workflows that automate staging data in and out of the cluster. Enable jobs to process larger amounts of data than could be run with the cluster allocated storage allotment.
- Automate deployment so staff can simply change files in their source repository to update the website
- Option to allow website users to run research software on their own data
- Option to support allowing website users to authenticate with Duke NETID sign in
- Option to deploy R shiny apps
- Expertise in front-end and back-end code
- Source Control
- to be able to undo current changes when software mysteriously stops working
- to enable multiple users to safely work on a project at the same time
- to track down changes that broke the code
- to provide accurate history of code
- for precise versioning to avoid being unsure what version someone used
- Automate tests - customers will receive an email if a change breaks the software
- Automate deployment of websites, container builds, etc.
- Create relational databases