NMDOT Traffic Safety Division Data System - Archived Data Management Functionality
Subsystem Description
The Archived Data Management Subsystem (ADMS) collects, archives, manages, and distributes data generated from ITS sources for use in transportation administration, policy evaluation, safety, planning, performance monitoring, program assessment, operations, and research applications. The data received is formatted and tagged with attributes that define the data source, conditions under which it was collected, data transformations, and other information (i.e. meta data) necessary to interpret the data. The subsystem can fuse ITS generated data with data from non-ITS sources and other archives to generate information products utilizing data from multiple functional areas, modes, and jurisdictions. The subsystem prepares data products that can serve as inputs to federal, state, and local data reporting systems. This subsystem may be implemented in many different ways. It may reside within an operational center and provide focused access to a particular agency's data archives. Alternatively, it may operate as a distinct center that collects data from multiple agencies and sources and provides a general data warehouse service for a region.
Functional Area: ITS Data Repository
Collect and maintain data and data catalogs from one or more data sources. May include quality checks, error notification, and archive coordination.
ID | Requirement | Status |
---|---|---|
1 | The center shall collect data to be archived from one or more data sources. | Planned |
2 | The center shall collect data catalogs from one or more data sources. A catalog describes the data contained in the collection of archived data and may include descriptions of the schema or structure of the data, a description of the contents of the data; e.g., time range of entries, number of entries; or a sample of the data (e. g. a thumbnail). | Planned |
3 | The center shall store the archived data in a focused repository that is suited to a particular set of ITS data users. | Planned |
4 | The center shall include capabilities for performing quality checks on the incoming archived data. | Planned |
5 | The center shall include capabilities for error notification on the incoming archived data. | Planned |
6 | The center shall include capabilities for archive to archive coordination. | Planned |
7 | The center shall support a broad range of archived data management implementations, ranging from simple data marts that collect a focused set of data and serve a particular user community to large-scale data warehouses that collect, integrate, and summarize transportation data from multiple sources and serve a broad array of users within a region. | Planned |
8 | The center shall perform quality checks on received data. | Planned |
9 | The center shall provide the capability to execute methods on the incoming data such as cleansing, summarizations, aggregations, or transformations applied to the data before it is stored in the archive. | Planned |
10 | The center shall respond to requests from the administrator interface function to maintain the archive data. | Planned |
11 | When data or a catalog of data is received from the archive, the center shall generate the requested data product for the users systems. | Planned |
Functional Area: On-Line Analysis and Mining
Advanced data analysis and mining features to support discovery of information, patterns, and correlations in large ITS archives.
ID | Requirement | Status |
---|---|---|
1 | The center shall support the interface with Archive Data User Systems for requests for analysis of the archive data. | Planned |
2 | The center shall provide the capability to perform activities such as data mining, data fusion, summarizations, aggregations, and recreation from archive data. This may include multidimensional analysis, selective summarization and expansion of data details, and many other advanced analysis services. | Planned |
3 | The center shall receive the user’s systems requests and develop the request to retrieve the data from the archive. | Planned |
4 | The center shall respond to users systems requests for a catalog of the archived data analysis products available. | Planned |
5 | For archive analysis and data mining products requiring financial payment the center shall process the financial requests and manage an interface to a Financial Institution. | Planned |
Functional Area: Virtual Data Warehouse Services
Provides access to data from geographically dispersed archives and coordinates information exchange with a local data warehouse. Also provides the specialized publishing, directory services, and transaction management functions associated with coordinating remote archives.
ID | Requirement | Status |
---|---|---|
1 | The center shall provide capabilities to access "in-place" data from geographically dispersed archives. These capabilities may include analysis, data fusion, or data mining. | Planned |
2 | The center shall coordinate information exchange with a local data warehouse. | Planned |
3 | The center shall provide the specialized publishing, directory services, and transaction management functions associated with coordinating remote archives. | Planned |
4 | The center shall support the collection of archived data from other archives on an as-needed basis. (This minimizes the need to duplicate the comprehensive set of data from the remote archives in the local data warehouse.) | Planned |
5 | The center shall use data collected from different archives to build a set of global schema including the data archive definitions for the local archive plus any archives known to the local archive. | Planned |
6 | The center shall provide the local archived data schema to other archive systems. | Planned |