#MOCKAROO DATA GENERATOR CODE#

Adding/removing resources to load would require modifying Java code (SourceFile.java).

Large - either need to figure out how to wait until system is fully up to get auth token, or refactor code to not use APIs, which would require getting resource repository beans (and waiting long enough so that the beans exist).

Requires access to authentication/authorization and configuration for that, since it uses APIs.

Disadvantages over current approach, with estimate of disadvantage.

Small - we don't need flexible configuration, as long as the CSV files have the exact demo data we want to load into the system.

Flexible in configuring CSV files for import, through mapping files.

Small - we don't care about updating resources, as demo data is generally assumed to be added to a 'fresh' system.

Configurable if resources are allowed to be updated.

Small/Medium - mapping is useful, but we don't need configurable mapping, as long as we can map data to each other by UUID.

Supports mapping between resources, which field is configurable (by id, code, etc.).

Once it is established, it shouldn't need to be constantly validated.

Small - Demo data errors can be fixed.

Data validation, not sure extent of validation.

Medium - this is better than editing JSON.

Updating data is relatively simple, editing the CSV files.UUID generation is handled by Java code, since it uses APIs.Advantages over current approach, with estimate of advantage (small, medium, large).Flexible in configuring CSV files for import, through mapping files (column names don't have to match database, can use a default value instead of a value in the CSV file, can skip columns).Does data validation as well as data loading, not sure what this validation encompasses.Uses APIs to create/update resources (not tables directly).Loads CSV files defined in SourceFile.java to create/update to corresponding resources.Is currently a separate Spring Boot application that runs when the OpenLMIS system has finished starting up.Incorporate RefData seed tool into each service that needs demo data.Options for a New Approach Option 1: Merge RefData Seed Tool into services

#MOCKAROO DATA GENERATOR FULL#

In recent adding of new vaccine demo data, it took one developer about 3-4 full days to do, with more than half of that time dealing with the first two pain points.

Virtually impossible for a non-developer to add/edit demo data because the JSON format is hard for a human to read/parse.

Which means that demo data loading might fail on service startup because of foreign key issues. Any JSON files that are not specified in this JavaScript file will be loaded into db non-deterministically.

In demo-data folder, we have a generator.js file that specifies JSON file order to load into db.

Adding a new JSON file (for a new table), need to make sure loading order is correct.

Manually need to ensure there are no foreign key violations.

Most tools we use (pgAdmin, psql, Mockaroo), can easily export to CSV, but not to JSON.

Needing to convert from (CSV) format to JSON to add to our demo data files.

Needing to generate own UUIDs when adding JSON ().

Flyway picks up this SQL file and runs it after all migrations on service startup, if Spring profile demo-data is set.

On gradle task (demoDataSeed), these JSON files are turned into a SQL file that has INSERTs on a (potentially large) JSON blob.

Current demo data loading approach (for each microservice).

Recently designed and implemented new demo data (for EPI/vaccines) to load on startup for each service (see Vaccine Demo Data).

How to wait until services are available before trying to add demo data.

This option has not been researched in depth this is just an initial guess There's some maintenance to keep demo data synced in Github and Mockaroo Demo data is in ubiquitous export format (CSV, not JSON)ĭeveloper to add/edit demo data (maintenance)Įmbed RefData Seed Tool into each microserviceīecause generated UUIDs would be different each time, unknown if any integration or contract tests depend on certain UUIDs

Mockaroo data generator