How QDAP can obtain data from Queensland Government agencies

This use case demonstrates how the Queensland Data Analytics Platform (QDAP) can obtain data from Queensland Government agencies.

The flowchart provides an overview of the steps required for each party to fulfill their obligations throughout the process. The flowchart is limited in its focus and surrounding steps or information can be found in the workflow below. The workflow tables provide practical information which will support decision makers throughout the data sharing process. This work aims to provide data custodians, recipients and users with clarity around what they need to do when sharing and using data.

Workflow

QDAP engaged and gap identified

The requesting agency engages QDAP to undertake integration, linking, or analytics work. For example, the COVID-19 Taskforce wishes to send SMS messages to people under quarantine.

Activities Supporting DocumentsResponsibilities
Identify opportunity
Assess risks and data quality issues
Review legislation for restrictions/enablers
Address privacy and security requirements
Information Sharing Authorising Framework (ISAF)
 
Privacy Impact Assessment (OIC)
 
Information Privacy Act
 
Agency specific legislation
Requesting agency needs to clearly understand what they want to do with the data

QDAP needs to determine if they have capacity and the expertise to deal with the request.

QDAP then identifies the need for additional data and initiates a request for data for a specific purpose. For example, data required from QH regarding people under quarantine to allow an SMS to be sent.

Activities Supporting documentsResponsibilities

Review current data holdings and permitted usage conditions of that data. If data is currently not held by QDAP, then determine which agency holds the required data and complete a request for the data.

The request must include specific details about the data requested but also general information about how QDAP will:

  • Monitor and audit
  • Control access
  • Provide options for linkage and creating new datasets
  • Publish details of the dataset to their catalogue/WoG catalogue
  • Potentially publish outputs (e.g. visualisations) to the COVID19 website
  • Manage recordkeeping/destruction
  • Potentially publish deidentified and/or aggregated datasets to Open Data
Data catalogue of QDAP holdings including permitted usage conditions

Data catalogue of Queensland Government data holdings including custodian details *

Open data catalogue (including spatial and ABS etc.)

QDAP to determine whether data is available and if the use meets the permitted usage conditions. It can then identify additional data sources if needed.

The Data Governance Body is an escalation point, available to resolve issues around permitted usage.

* To assist with the discovery of data and information for agencies that do not have the ability to publish the availability via a formal catalogue, an interim Data Catalogue has been established. Queensland Government users can search this register to locate data and information that may be of use.  It can be found at the Queensland Government Data, Information Sharing and Analytics collaboration site. For access, please contact transformation@qgcio.qld.gov.au.

Supplying agency to consider request

The supplying agency determines whether it holds the requested data and undertakes an assessment of the data request. For example, QH reviews its information asset register and determines who is the Information Asset Custodian and the DG has the authority to release the data.

If the agency cannot provide the data, QDAP will need to return to the identify and request stage or escalate to the governance body if dissatisfied with agency’s assessment.

ActivitiesSupporting DocumentsResponsibilities

Supplying agency undertakes a review of data holdings and determines if it holds the required data, who the custodian is, and who has the authority to release.

If the information is available, the supplying agency undertakes an assessment of the data against a range of criteria, including:

  • Quality
  • Accuracy
  • Timeliness
  • Format
  • Effort/time to extract
  • Security classification
  • Personally identifiable information
Agency Information Asset Register

Queensland Government Information Security Classification Framework (QGISCF)

Information quality framework guideline
Information asset custodian is to provide detailed information regarding the dataset and its condition and advice as to whether it is fit for purpose (i.e. if it can meet the requirements of the request).

Authorisation of data sharing

The supplying agency's authorising officer must evaluate the preliminary assessment against the request and make a determination as to whether the data should be released.

ActivitiesSupporting documentationResponsibilities

If the request is rejected, the authorising officer is to provide reasons. If the request is approved, the following statements must be included in the authorisation:

  • statement of authorisation and the legal instrument being relied upon (e.g. the NPPs and IPPs in the Information Privacy Act 2009 (Qld), the Public Health Act 2005 (Qld), or the Hospital and Health Boards Act 2011 (Qld))
  • timeframe
  • purpose of sharing
  • permitted usage conditions
  • access limitations
  • security classification
  • availability of data and the outputs of the analytics, including publication permission (e.g. of aggregate datasets to Open Data)
  • process to follow if further use of data is required for another purpose
  • monitoring and auditing requirements
  • record keeping and destruction protocols, including a review period to renegotiate with custodians.

The authorisation must then be communicated to QDAP. This includes the above statements, and any conditions of use beyond those. It can be communicated via proforma email, a web form available through QDAP, an MSA or an MOU. A record of the authorisation is then to be stored and published to the QDAP master list.

Information Sharing Authorising Framework (ISAF)

Master sharing agreement

MOU

Proforma email

QGCIO page 'How do I get approval to share?'

Authorising officer is to ensure that due process has been followed and that the data is suitable for release. This may require additional advice for some sensitive datasets from agency legal, privacy or security officers.

The Data Governance Body may wish to review the reasons provided if the request is rejected.

Access and use of data

Once the authorisation phase is complete, the supplying agency and QDAP determine the mechanism for data to be shared. The data is then shared. For example, QH data custodian supplies requested data to QDAP as per the timeframe(s) in the permitted usage conditions.

ActivitiesSupporting documentsResponsibilities

Mechanism determined based on the security classification and sensitivity of the data and its format.

Data can then be transferred to QDAP at the time and in the format agreed upon.

A record of the sharing of data is to be kept and published for the purpose of transparency.

Permitted usage conditions *

QGISCF

Information exchange schedule
Information asset custodian arranges transfer of data and ensures that timeframe conditions are met.

QDAP receives data and stores / restricts access in accordance with the permitted usage conditions.

Data Governance Body periodically audits authorisations to ensure permitted usage conditions are being met.

Once the data is received, QDAP reviews and prepares the data to ensure it is fit for purpose.

ActivitiesSupporting documentsResponsibilities
Depending on the state of the data supplied and the requirements of the request, this may involve, but is not limited to, activities such as quality assurance, cleansing, and applying unique identifiers for data matching. Original data request

Permitted usage conditions *
QDAP to complete data review and preparation and liaise with supplying agency to resolve issues.

Data Governance Body to resolve any issues with the supplying agency if required.

* To assist with the protection of data and information shared to support COVID-19 related work, a Permitted usage authority register has been established. Queensland Government users can search this register to find out the permitted usage conditions of data and information shared through the Data Catalogue.  It can be found at the Queensland Government Data, Information Sharing and Analytics collaboration site. For access, please contact transformation@qgcio.qld.gov.au.

Once the data is determined to be fit for purpose, it can be used for its intended purpose. For example, data is provided to SSQ to support outbound COVID-19 communications.

ActivitiesSupporting documentsResponsibilities
This may involve data being used and accessed within the confines of QDAP as well as QDAP processing data and providing it to other Queensland Government agencies to use or consume.

QDAP may publish data to its Data Warehouse or the [interim] Data Catalogue.
 QDAP provides requested data to users as per the terms of the original request.

Data users are aware of the conditions of use and use data only for its intended purpose(s).

Data Governance Body has oversight of the process, reviews outcomes, makes suggestions for improvements and deals with any issues escalated to it.

Data is to be published as per its permitted usage conditions.

ActivitiesSupporting documentsResponsibilities
QDAP publishes to appropriate and agreed repositories in accordance with the conditions specified in the original request and the agreed permitted usage conditions.

QDAP to add record to their Data Warehouse and/or the [interim] Data Catalogue.

QDAP to schedue destruction of data according to permitted usage conditions and timeframe.
Public Records Act

Records Governance Principles

Open data policy statement

Information access and use policy
QDAP ensures that authorisations are kept for recordkeeping purposes and available for auditing.

Data Governance Body to provide oversight and review capabilities as well as deal with escalated issues.

Data management and governance

Access to data must be managed appropriately.

Activities Supporting documentsResponsibilities
QDAP has documented, automated processes in place to ensure that only those who are permitted to access and use the data have access and that permitted usage conditions are adhered to.
Information access and use policy

Audit schedule

Access notifications at log in to ensure users are aware of inappropriate access and use penalties (as per CCC Operation Impala Recommendations).
Requesting agency provides permission/endorsement for their staff PRIOR to requesting access to QDAP.

QDAP ensures appropriate user access permissions are provided and kept up to date, and that users are aware of their responsibilities.

There must be a body in place to oversee governance. For example, the Data Governance Body.

Activities Supporting documentsResponsibilities
There is a group in place to oversee the process, deal with escalated issues, resolve problems and make determinations. 

Ideally this body is comprised of cross agency representatives from key data supplying or requesting agencies with the seniority and authority to act.
  Responsibilities of this body may include managing by exception:
- Resolving unforeseen issues
- Deciding on access requests
- Dealing with rejected requests for data to be added to QDAP
- Data not being provided as specified (e.g. quality, timeliness etc.)
- Amendments to conditions of use
- identifying additional data sources
Dealing with adverse audit results (e.g. unauthorised access, use or release).

Data is to be retired or destroyed appropriately.

Activities Supporting documentsResponsibilities
Once the permitted usage timeframe has ended, the data should be destroyed unless other arrangements have been made. Permitted usage conditions *

Public Records Act

Records Governance Principles

Retention and disposal schedule
QDAP to regularly review data holdings and determine if data has reached the end of its retention period.

Supplying agency to request and/or approve destruction, or provide reasons to retain.

* To assist with the protection of data and information shared to support COVID-19 related work, a Permitted usage authority register has been established. Queensland Government users can search this register to find out the permitted usage conditions of data and information shared through the Data Catalogue.  It can be found at the Queensland Government Data, Information Sharing and Analytics collaboration site. For access, please contact transformation@qgcio.qld.gov.au.


Last Reviewed: 09 June 2020