DataDriven

Managing Data for the City of New Orleans

The City of New Orleans has adapted these resources from the City and County of San Francisco and the DataSF initiative. The purpose of this information is to provide instruction to Data Coordinators within The City of New Orleans. Data Coordinators should use these resources to help them in their new role. We’ll update this guide as the roles and responsibilities of the Data Coordinator evolve and as we learn more about implementing DataDriven NOLA.

Reference

Appendix: Templates

Appendix A1. TEMPLATE: Data Inventory (Excel)

Appendix A2. TEMPLATE: Data Inventory (Online form)                         

Appendix A3. Data Inventory: Fields and Descriptions

The data inventory template includes the following fields. 

Field Description
Department/Division What department collects and manages the data as an official record?
Dataset name Brief descriptive name for the dataset
Brief description of data Include a brief description of the dataset. What is the purpose? What is it used for? Include key data fields if possible.
Data Coordinator (Business knowledge) - Name Who manages the data and is responsible for making changes to the data? Who understands what the dataset includes and can answer questions about it?
Data Coordinator (Business knowledge) - Email Please enter the email of the Data Coordinator.
Data Custodian (Technical knowledge) - Name If applicable, include a contact who manages the technical execution of the database (e.g. database management, access and extraction).
Data Custodian (Technical knowledge) - Email Please enter the email of the Data Custodian.
Data source What information system or database contains the data? Or what shared server or shared drive contains the data?
Start date How far back does the data go? Use format (MM/DD/YYYY)
End date When does the data end? Use format (MM/DD/YYYY). If the data is still being updated, use "Current".
Geographic coverage Does this data cover the whole city or a subset? Or does it have broader coverage than the city?
Geographic granularity What is the lowest level of geography in the data? For example, if the data is collected by address, it would be Street Address.
Frequency of data change At what rate does the information in the dataset change?
Number of records How many records or entries does the dataset include?
Format What format is the data in? e.g. excel, sql, oracle database, pdf, word, etc.
Existing ETLs Are there existing database connections or extractions?
Existing publication Is this data already published (made publicly available) in some form or another, e.g. a report.
Link to existing publication If you answered yes to the previous question, please include a link to the report or document.
Priority/value What is your sense of the relative value in publishing this data?
  • High - Existing and ongoing requests for this data; this data addresses pressing information needs or pain points (within or without the city); or we have heard compelling examples of how this data could be used
  • Medium - This data may be useful for other departments or for people external to the city; we occasionally receive requests for this information; or we have heard some examples for how this data could be used
  • Low - This data has unclear value for either the public or other city departments; we have never received requests for this data; or we have never heard a use case for this data
Priority/value comments Any details or comments to add to your response on priority?
Technical challenges What technical challenges, if any, do you anticipate in publishing this data?
Data Classification How would you classify this data?
  • Public - this data could be publicly disseminated without any concerns
  • Protected - this data is protected by law or regulation and can only be shared or accessed internally and per organizational procedures; OR this information includes individually identified information
  • Sensitive - in its raw form, this data poses security concerns, could be misused to target individuals or poses other concerns.
Protected - Details If you marked “Protected” for Data Classification, please indicate what law(s)/regulation(s) protect this data.
Sensitive - Details If you marked “Sensitive” for Data Classification, please describe your concerns
Data Quality Do you have concerns about the quality of this dataset?
Data Quality - Details Please describe your concerns with the quality of this dataset.

Appendix B. EXAMPLE: Completed Safety and Permits Inventory

Appendix C. Definitions

Term Definition
Dataset

Contents of a single database table, worksheet or defined view; data is provided as a single combination of unique rows (or records) and corresponding columns (or fields) describing that row.

Example - Database: A database may contain several data tables - each data table constitutes a dataset. However, you could also create new datasets by combining data from different tables into a new table.
Data Schema or Standard Specification that defines the structure of the data (i.e. required data elements and types and supporting definitions)
Data source Technology or system that stores data, including databases, named spreadsheets, information systems, business applications, etc.
ETL Extract, Transform, Load -  three database functions that are combined into one tool to pull data out of one database and place it into another.
  • Extract - process of reading data from a database.
  • Transform - process of converting the extracted data from its previous form into the form it needs to be in so that it can be placed into another database.  (rules, lookup tables, combining with other data)
  • Load - process of writing the data into the target database.
Geospatial data Data related to the position of things in the real world, including boundaries or locations
Metadata Descriptive information about a dataset
Tabular Data that is presented in columns or tables
Taxonomy or Category Methodology by which items or datasets are classified or grouped under a similar theme or topic

Appendix D. Resources & Credits

This guidebook was originally created by The City and County of San Fransciso using input from a number of resources, including:

Title Attribution License
City of Philadelphia Open Data Guidebook City of Philadelphia, Office of Innovation and Technology Creative Commons Attribution-ShareAlike 4.0 International license
New York State Open Data Handbook New York State Open Data Initiative  
Open Data Handbook Open Knowledge Foundation  
Sunlight Foundation Open Data Guidelines Sunlight Foundation