Welcome to PACE

Protected Analytics Computing Environment

Protected Analytics Computing Environment (PACE) is a highly protected virtual network space that serves as a marketplace where approved users can work with identifiable protected health information. PACE simplifies the effort of obtaining EHR (Electronic Health Record) data from Duke Health enterprise data warehouse and Duke's Maestro Care (Epic) EHR system, while supporting collaborators from around the world with approved NetIDs. The marketplace offers a rich set of tools, services, and resources required by research and quality initiatives. Within the protected enclave, PACE users are provided the ability to select operating systems, analytic tools (e.g. R, SAS, Python), services (e.g. an Honest Broker or Transfer Agent service to securely release data outside of PACE), compute and data sources.

PACE Fees

Service Cost Charge Type
New PACE Workspace $627.25

Per person

New PACE Workspace (Student) $409.20 Per person
New Shared Project Folder $280.00 Per project
Honest Broker $120.00 Per export by Central Honest Broker
Transfer Agent $225.00 Per person for a project; ability to execute multiple audited exports, without using an Honest Broker
Background Check $50.00 (approx) per background check, if not already on file with Duke Human Resources

 

CTSA image

 

PACE Prerequisites

Information on PACE Account Requirements

If you have questions about CoreResearch@Duke, there is additional information here: CoreResearch@Duke Training Resources

PACE Account Requirements

  1. Complete PACE Training for Duke Employees and Affiliates in LMS
  2. Obtain a CoreResearch@Duke (see below) Service Request ID.
  3. Submit PACE Account Request
    • Provide IRB, RPR, QI, or equivalent number at time of request (requester must be Key Personnel on an IRB study with an approved or exempt status) 
  1. Complete PACE Training for Duke Employees and Affiliates in LMS
  2. Complete HIPAA Privacy and Security Training in LMS
  3. Request a waiver of PACE fees. Please note – a waiver for student access fees is only available if the associated project is unable to cover the cost for the student’s access to PACE.
    • Complete this survey.
    • You will receive an email from the PACE Team when your waiver is approved. The email will include a Requisition ID (REQ ID) that you will need to include in your PACE Account Request.
  4. Submit PACE Account Request
    • Provide IRB, RPR, QI, or equivalent number at time of request (requester must be Key Personnel on an IRB study with an approved or exempt status)
  1. Establish a DHE account – Need a sponsor in the Health System to request a DHE account on your behalf
  2. Complete PACE Training for Duke Employees and Affiliates in LMS
  3. Complete HIPAA Privacy and Security Training in LMS
  4. Obtain a CoreResearch@Duke (see below) Service Request ID.
  5. Submit PACE Account Request
    • Provide IRB, RPR, QI, or equivalent number at time of request (requester must be Key Personnel on an IRB study with an approved or exempt status)

How to obtain Core Research Request ID

Surgery and Head & Neck Surgery Users

Employees and students affiliated with the Departments of Surgery and Head & Neck Surgery must submit a request for PACE services through the Section’s data services intake system: Section of Surgical Disciplines Data Service Requests

Hospital Cost Centers (Company Code 030)

Employees and students with projects funded by Hospital Cost Centers will, for now, be invoiced manually for PACE Services. For more information, please contact PACE Support via Service Now.

Obtaining REQ-ID through CoreResearch@Duke

TL;DR

  1. Obtain REQ-ID: If you are authorized to request billable services, you can obtain your REQ-ID through CoreResearch@Duke. Take a note of this number, generated in the format of REQ-00000#####.
    1. This Request ID is required by the PACE team to bill services rendered. 
    2. If you do not know who is authorized to request billable services for your project, please contact your PI, a delegate of your PI, your Financial Manager, Grant Manager, Research Project Manager, Departmental Business Manager, or mentor. For steps, see: Add Facility User to Fund Code link
    3. Click here to watch a video tutorial that explains and illustrates how to order services through CoreResearch.
  2. PACE Service Provision: Once the above steps are completed, the PACE Team will be able to provide you with the requested services requested via ServiceNow - PACE Service Request Form.

Please follow these steps to ensure a smooth and efficient process.

Step-by-Step Instructions

  1. Log in to coreresearch.duke.edu

  2. Select Manage Service Requests. (If you do not see the green Manage Service Requests button, please look in the lower right corner of the page and select the PI, PI Delegate, or Facility User role.)
    1. Click Add New Request.
    2. Enter the request Title (to identify the project or request; often, the IRB number).
    3. Select the "Protected Analytics Computing Environment (PACE) Service Center" (Core).
    4. Select the Project ID. (Click the arrow to the right to search. In the projects search, if you have your fund code, click the Search by Query arrow and select By Account#.)
    5. Click Save.
  3. Click Add by Group (not Add New).
    1. Select the PACE Services.
    2. Click Save.
    3. Click Submit for Review. (When complete, you will see the message Operation Successful.)
  4. Copy, or make a note of, the generated REQ-00000##### number. You will need this to Request PACE Services from the PACE Team. 

Notes

  • The PI and a Financial Manager are automatically created in CoreResearch@Duke as assigned in SAP, Duke’s accounting system.
  • To see the roles assigned to you, log in to CoreResearch@Duke, and look at the bottom righthand side of the screen.
  • To see your projects, click Manage Projects from the main menu.
  • To return to the main menu, click the word “CoreResearch@Duke” at the top left.
  • To assign a PI Delegate, see: Assign a PI Delegate.
  • There is more help, with screenshots, here: Facility User guide for PACE.
  • Click here to watch a video tutorial that explains and illustrates how to order services through CoreResearch.

Getting Help

Resources

Core PACE Resources

Link Description Accessible from
Data Portal A data transfer portal to import data into PACE Inside/Outside PACE
Group Manager Manage access to your PACE Project folder Outside PACE
DEDUCE An online query tool Inside/Outside PACE
PACE Gitlab Web-based Git repository Inside PACE

Honest Broker/TA

Information on Honest Broker and Transfer Agents

Requesting Honest Broker Services

  1. Place your data to be exported in your PACE Outbox 
    • This folder is located on the left hand side of the PC drive listing in your Folder Navigator in PACE.
    • Your personal outbox will be listed under the Drive location entitled “PACE Outbox (O:)
  2. Click the button for "Honest Broker Service Request" above
  3. Fill in relevant information on the form including:
    • Honest Broker Type (e.g., PACE)
    • Location of Files to Be Transferred (i.e. which file or folder in your Outbox within PACE)
    • IRB Number
    • CoreResearch Req ID
    • Specific Nature of Data Being Exported (i.e., is there PHI included in your export, tables, figures, code, datasets?)
  4. Once you select “Request Now” a ticket will be generated and sent to the Honest Broker for review and approval of your data for export.
  5. The Honest Broker will review data, and the IRB protocol where necessary.
  6. Once the data are approved for export, you will receive a secure link in an email from the Honest Broker to obtain your data.
    • The URL will expire in 76 hours

REDCap Data Exports to PACE

REDCap export is not permitted for PACE project study personnel

To export data from REDCap, please submit a Honest Broker service request including the following information:

  • Put “N/A: REDCap export” in the CoreResearch REQID field.
  • The REDCap PID.
  • The format in which you would like the data exported.
  • PACE Shared folder name. A separate folder within the Shared Project folder will be created for the exported data.

Please note that this is currently not charged against the project team. PHI can be exported from REDCap for work inside of PACE.

Software

Windows

Due to the specific licensing requirements for using Adobe Acrobat Pro software, persons wishing to use this product must have special access to the shared drive where the Adobe Acrobat Pro installation files are stored.

If you do not possess an Adobe Acrobat Pro license, you may obtain one by purchasing it through the Duke OIT software site.

Once you have obtained a license, or if you already possess one, please contact PACE Support via Service Now with your license information. The emailed receipt you receive after purchase can serve as proof of license.

Activation instructions

  1. Do not sign in with an Adobe ID.
  2. Instead, click "Sign in with an Enterprise ID"
  3. In the box for e-mail address or domain name, type netID@duke.edu, where netID is your Duke netID
  4. Press enter, press tab, or click the sign in bar
  5. You will be redirected to the Duke authentication page to enter your netID and password
  6. You will be redirected back to a page that indicates that you have activated Acrobat Pro DC.

General Information

The program may ask you to sign in periodically. Make sure to always sign in as directed above. Duke accounts are Enterprise ID's rather than Adobe ID's.

Our license does not include any cloud services or mobile application access. There are places to sign in within the application through which you will get an "access denied" error message. You still have the full version of the program, just not the cloud sync or add-on products for which Duke is not licensed.

The AQT installation file available is a free trial version. The AQT trial version is fully functional and has all the features you'll find in the registered version, but it is limited to a trial period of 30 days.

If you decide to purchase a license at the conclusion of the trial period, you may do so by navigating to the Allround Automations website at querytool.com/purchase.html on a computer outside of PACE. Once purchased, you will be able to enter your license code on the version installed in your PACE workspace.

After installing the Git software, please run the following command in the git bash command line:

  • $ git config --system http.sslverify false

Due to the specific licensing requirements for using JMP software, persons wishing to use this product must have special access to the shared drive where the JMP installation files are stored.

If you already possess a JMP license, please contact PACE Support with your license information.

If you do not possess a JMP license, please obtain one from Duke Software.

Due to the specific licensing requirements for using MATLAB software, persons wishing to use this product must have created an account for MATLAB through OIT Software.

If you already possess a MATLAB account, navigate here: https://in.mathworks.com/academia/tah-portal/duke-university-30359028.html (this is accessible within PACE). Then download the desired version and install.

If you do not possess an MATLAB account, you may obtain one it through Duke Software.

Due to the specific licensing requirements for using NVivo software, persons wishing to use this product must have special access to the shared drive where the NVivo installation files are stored.

If you already possess a NVivo license, please contact PACE Support with your license information.

If you do not possess an NVivo license, you may obtain one by purchasing it through Duke Software.

The PL/SQL Developer installation file available is a free trial version. The PL/SQL Developer trial version is fully functional and has all the features you'll find in the registered version, but it is limited to a trial period of 30 days.

If you decide to purchase a license at the conclusion of the trial period, you may do so by navigating to the Allround Automations website at https://www.allroundautomations.com/products/pl-sql-developer/ on a computer outside of PACE. Once purchased, you will be able to enter your license code on the version installed in your PACE workspace.

Configuring PIP

  1. Add Python to your Windows PATH. (See instructions below for how to do this step.)
  2. Open a Windows command prompt (click the magnifying glass at the bottom left, type cmd, and click on Command Prompt) and enter the following command to ensure pip is enabled: python -m ensurepip
  3. Copy the file dhcert.pem from S:\dhcert.pem to your local PACE C: drive.
    1. Open a Windows command prompt as administrator (click the magnifying glass at the bottom left, type cmd, right click on Command Prompt, and click Run as administrator).
    2. Enter the following command: copy "S:\dhcert.pem" C:\dhcert.pem
  4. Upgrade pip.
    1. Open a Windows command prompt as administrator (click the magnifying glass at the bottom left, type cmd, right click on Command Prompt, and click Run as administrator).
    2. Enter the following command: python -m pip install --cert C:\dhcert.pem --upgrade pip
  5. Install the Python package(s) you need.
    1. Open a Windows command prompt as administrator.
    2. Enter the following command, replacing [package_name] with the name of the package you need to install: python -m pip install --cert C:\dhcert.pem [package_name]

Add Python to Windows PATH

Python 2.7

  1. Open System Properties (right-click This PC on the desktop and select Properties)
  2. Click Advanced system settings in the sidebar.
  3. Click Environment Variables...
  4. Select Path in the System variables section
  5. Click Edit
  6. Add Python's path to the end of the list by clicking the New button and then entering the installation path for Python. For example:

    C:\Python27

  7. Please note that the above path is the default install location, but if you chose a different location during install, that is the path you will need to add on the above step.

Python 3

  • The best way to ensure Python 3 is in your Windows PATH is to elect to add it during the initial installation.
  • On the initial install window, make sure the box at the bottom to add Python to PATH is checked:

R Packages are accessible via only the Duke CRAN mirror.

Specify the Duke mirror in one of three ways:

  1. Within RStudio, select a mirror from the pop-up list without specifying the CRAN mirror - install.packages ("package_name_here") – or click “Packages -> Select CRAN Mirror” on the top menu bar.
    1. A list of available mirrors will be presented.
    2. Choose the last item in the list: "http mirrors"
    3. In the list of http mirrors, choose the Duke mirror: "USA (NC)"

  2. Specify the Duke CRAN mirror alongside the package name:

    install.packages ("package_name_here", repo="http://archive.linux.duke.edu/cran/")

    Note: Make sure to replace package_name_here in the above example with the name of the package you want to install.

  3. Set the default mirror by adding a setting to your .rprofile settings file:

    local({ r <- getOption("repos")

    r["CRAN"] <- " http://archive.linux.duke.edu/cran/”

    options(repos = r) })

If you need an R package that is not in CRAN (unusual):

  1. From outside PACE, copy the file to your PACE workspace by using the pacedata portal (https://pacedata.duhs.duke.edu)
  2. Log into PACE and locate the file in your pace home drive (this is where the pacesoftware portal drops files).

Due to the specific licensing requirements for using Stata software, persons wishing to use this product must have special access to the shared drive where the Stata installation files are stored.

If you already possess a Stata license, please contact PACE Support with your license information.

If you do not possess an Stata license, you may obtain one by purchasing it through Duke Software.

Linux

To be sure your system is up to date, run the following command in terminal:

  • sudo apt update && sudo apt upgrade -y

The preferred method to install most software packages in Ubuntu Linux is to use APT, Ubuntu's built-in, command line based package manager.

To use apt to install software on a PACE Ubuntu VDI, follow the steps below:

  1. First, open a terminal window. To find your terminal, you can click the Ubuntu logo at the top left of your desktop and start typing terminal into the search bar. When it appears, click the terminal icon to open it.
  2. Enter the apt install command as root at the $ prompt, like so: sudo apt install [insert_package_name_here], where [insert_package_name_here] is replaced with the name of the package in the Ubuntu repository.
  3. You may now be prompted to enter your NetID password. Please do so.
  4. Follow the onscreen prompts to complete the installation.
  5. If the install completes successfully, you should be able to start the application from the command line.
    1. Should the install fail, run: sudo apt --fix-broken install, before attempting to reinstall

 

To install and configure Anaconda, run the following commands:

  • bash ~/applications/Open\ Source/Linux/Anaconda/Anaconda3-2023.09-0-Linux-x86_64.sh
  • sudo cp ~/applications/dhcert.pem ~
  • conda config --set ssl_verify dhcert.pem

Install OpenBUGS

Run the following commands in terminal:

  1. sudo apt install libc6-dev-i386
  2. cp ~/applications/Open\ Source/Linux/OpenBUGS/OpenBUGS-3.2.3.tar.gz ~/.
  3. tar zxvf ~/OpenBUGS-3.2.3.tar.gz
  4. cd ~/OpenBUGS-3.2.3
  5. ./configure
  6. make
  7. sudo make install

Run OpenBUGS

Run the following command in terminal:

  • /usr/local/bin/OpenBUGS

Install R

Do the following, in terminal, to install R:

  1. Edit "/etc/apt/sources.list" file to add Duke CRAN library
    1.  sudo nano /etc/apt/sources.list
      1. Add the following line to the end of the file: deb https://archive.linux.duke.edu/cran/bin/linux/ubuntu focal-cran40/
  2. sudo apt update
  3. sudo apt install r-recommended

Configure R

Do the following, in terminal, to configure R:

Install RStudio

RStudio should install the following command: sudo apt install rstudio

NOTE: R must be installed beforehand

If that should fail, you can install RStudio from source, using the following command:

  • sudo dpkg -i ~/applications/Open\ Source/RStudio/rstudio-2023.12.1-402-amd64.deb

Follow the instructions below to install SAS:

  • Open a terminal window and enter the following commands.
  • cd ~/applications/Licensed/SAS/SAS9.4M8/software_depot
  • sudo ./setup.sh
  • Follow the onscreen prompts to complete the installation.

Software Request

To submit a software request click here.

Additional PACE Information

Get Help with PACE

Click here for Tipsheets

Research Data Repository & PACE

SAS Config for Oracle

The videos below provide training on accessing and using PACE. If you have trouble playing them in your web browser, please try using the download links below each video to watch offline in your video player of choice.

Video Subject Link
Requesting PACE Click Here
Launching PACE Click Here
Finding Software in PACE Click Here
Collaborating in PACE Click Here
Importing and Exporting Data in PACE Click Here
Requesting Honest Broker Services Click Here

 

PACE Information

For information about data classifications and Duke Services, click here

For a quick brochure on PACE and its capabilities and tools, click here

PACE Support Description and SLA

This support plan is for all PACE users across the institution. Support availability and response times will vary depending on the type and complexity of a request. The following table summarizes the offerings

Support Offerring Standard Support
Availability for General Inquiries Business Hours
Availability for Incident Reports Business Hours
PACE Initial Response Times Next Business Day
Web/Service Now Get IT Portal
Honest Broker Request: De-identified (Plots, Charts, Tables, etc.) Next Business Day
Honest Broker Request (Data): PHI or LDS that requires HB to review the study's approved IRB protocol Delivery time will depend on the complexity of the request.
PACE Account Provisioning with Compliance Checks (Background Check Not Included) Up to 2 Business Days
Duke Background Check (if not completed during HR onboarding) Up to 10 Business Days
Add software application to PACE Delivery time can vary significantly depending on the complexity of the security evaluation, and the licensing situation.
Fulfillment of project folder request 1 - 5 Business Days

Definitions

  • "Business Hours" are 8AM-5PM
  • "Business Days" are Monday through Friday, excluding Duke Holidays.
  • An "Incident" is an occurrence during which end users’ use of PACE services is adversely impacted.

User responsibilities

User is responsible for using PACE according to the training (follow this link for training). Delayed response from users to PACE Service Team communications will result in delayed response.

Support requests

Users submit support requests completing a PACE Service Now Request. Users will receive a system-generated response within minutes containing the ticket number and a direct link to the ticket.

Communications

Tickets

Communications between PACE support engineers and Users are conducted using the ticketing application, which maintains a time-stamped transcript of communications, and sends emails to Users and PACE staff as tickets are updated.

Response time

PACE support team shall use their best efforts to respond in a timely fashion.

Response times

PACE support team commits to acknowledging receipt of a support ticket within the next Business Day following submission of a support request.

Incident reporting

The PACE support team will respond to the report of an Incident by troubleshooting the cause(s) of the Incident and resolving them or providing workarounds, if possible.  Users should submit incidents either through Service Now (or Get IT Portal) or by dialing (919) 684-2243 and pressing 4 for Research Support and Navigation. 

Helpful Resources

PACE-Related Policies

Policies related to PACE can be found on the Duke Office of Clinical Research (DOCR) website.