Last Mile ETL: Enabling Fast, Flexible Data Delivery

GoodData recently launched its newest feature: Last Mile ETL, aka SQL-based datasets. This offers users a streamlined approach to data management, and allows for more efficient and effective data analysis.

Last Mile ETL is designed to simplify and speed up data delivery to end users. The feature fully supports making changes to your logical data model (LDM) within the GoodData interface — without affecting the data and data structure in your actual database. The newly created SQL datasets are not permanently stored in the database, but created on-demand to provide virtual views with read-level access.

A GoodData dataset is a collection of related data that has been organized and transformed to enable analysis and reporting. It can be created from one or more source data tables and is defined by its logical data model. The datasets provide a semantic layer that allows users to interact with data using business concepts rather than technical database structures.

Why Is Last Mile ETL Important?

Last Mile ETL provides several benefits to organizations that are looking to improve their data management capabilities. Here are just three of the main ones:

1. No need to build huge ETL/ELT pipelines to make data changes in your database. Users can easily create new datasets and tables, change data types, and split or merge attributes within the logical data model. This can be done within the GoodData interface without affecting the underlying data sources, providing a more flexible and efficient approach to data management. Note: this feature is great for PoC (Proof of Concept), but for production systems you should always build ETL/ELT pipelines.
2. The feature aligns with the developer community, where SQL knowledge is widely recognized and utilized. Developers can easily access and analyze data using familiar SQL queries. This enables them to customize their logical data models and reuse these changes in different visualizations to meet specific business needs.
3. GoodData enables non-technical users to easily access and analyze data through MAQL without requiring advanced technical skills. Last Mile ETL complements this self-service capability, and provides analytics engineers with the technical flexibility to build custom solutions and meet complex business requirements with SQL queries.

A key ingredient of the above benefits is velocity; Last Mile ETL accelerates the activities related to data management. It is no longer necessary to make a request to developers and wait for their response to carry out changes in the database. Instead of taking several days or weeks, new changes within the logical data model are implemented quickly and easily through GoodData's interface by creating new datasets using SQL queries.

Last Mile ETL makes it easier for business users to reuse new or modified attributes in the current logical data model and create metrics and appropriate visualizations for their future decisions. This leads to increased productivity and time-saving as the changes are immediately reflected in the data model – without the assistance of developers. It also increases profit as money is not lost due to delays. Ultimately, business users can improve how they work with data, leading to better decision-making and improved business outcomes.

How Can Last Mile ETL Improve Your Analytics?

Last Mile ETL can improve your analytics in several ways. Each revolves around the same core objective: providing changes to your logical data model using SQL queries, without these changes affecting your database.

The feature addresses two main issues that all companies face:

• The need to implement minor changes in the database without waiting a long time for developers
• The fact that most users only have read-only access to the database and cannot make changes

If you are an analytical engineer, you can use Last Mile ETL in the following ways.

1. Reuse your existing datasets

Convert your current datasets in the LDM to SQL datasets, and you can easily make changes such as concatenating or splitting attributes and changing data types or structures. For example, in the LDM you might have customer information as one attribute (name and email address in a single column). With Last Mile ETL, you can use SQL query to split the column into two separate columns for name and email address – making it easier to analyze and visualize this data. Similarly, if you need to combine attributes like latitude and longitude into a single attribute geo coordinate, you can easily do so with SQL queries.

2. Create new datasets

You can create new tables or even new attributes that are not in your database as SQL datasets. Based on your specific requirements, you can also combine different attributes from different tables into one newly created table, or alternatively, create different tables from one table. Creating new attributes is possible too (for example, making a new column for customer age; calculated as current year minus birth year).

3. Change your data structure in GoodData

Unsure whether data types in your current database will work with the GoodData platform? Using Last Mile ETL to adjust your logical data model in GoodData allows you to test out the platform and see if it's the right analytics solution for your needs. Simply make any necessary changes to the data types, structures, and attributes to ensure compatibility with the platform.  This saves time and effort as you won't need to modify your database.

For further technical insights on Last Mile ETL, we recommend reading Introducing Last Mile ETL: One Tool for Better Data Transformation, which allows you to dig deeper and follow practical examples.

Summary

GoodData's latest feature provides users with a streamlined approach to data management, resulting in more efficient and effective data analysis. More specifically, Last Mile ETL:

• Enables fast iteration with data models
• Aligns with the developer community
• Complements non-technical end users and analytics engineers
• Addresses pain points; such as limited database access and waiting for developers to implement minor changes
• Improves analytics by allowing both the reuse of existing datasets and the creation of new ones
• Users can make changes without modifying their databases
• Business users can improve how they work with data, leading to better decision-making and outcomes.

Try Last Mile ETL for Yourself

Want to make managing your analytics easier? Sign up for a free GoodData trial to discover how Last Mile ETL can help streamline your data analysis.