Astera Centerprise 8.0 – What’s New, What’s Fixed, and What’s Improved¶
Centerprise 8.0 is a major release in Astera’s line of ETL solutions and is shipped with new features, existing feature updates, as well as a brand-new UI. Here’s a short summary of what’s new, what’s fixed, and what’s improved:
Centerprise 8.0 – What’s New¶
REST Server Design¶
We are now shipping a REST-based Centerprise server with the release of version 8.0. We have transitioned from .NET remoting to RESTful web services.
Introducing Lean Centerprise client – A Modern, Scalable Architecture¶
You asked for an easier way to deploy Centerprise across a company network, and we delivered! This is good news for your IT administrators. We have introduced a lightweight version of the Centerprise client with the 8.0 upgrade. Assemblies and DLLs (Dynamic Link Libraries) for most database drivers now reside on the Centerprise server instead of the client machines. Now you can scale horizontally and add more clients to an existing cluster of servers with minimum IT overhead.
SmartMatch Functionality¶
We have introduced the SmartMatch functionality in Centerprise 8.0 to simplify complex mapping jobs in ETL flows. It uses a Synonym Dictionary File created by the user to resolve mapping issues that arise due to naming inconsistencies in the incoming source data. The Synonym Dictionary file is a simple txt file in which users can specify variations of the header name in a pipe delimited format and Astera integration server will automatically read and match field names at run-time based on the predefined information in the dictionary.
Data Cleanse Transformation¶
Data Cleanse is a new addition to Centerprise’s library of built-in transformations. Data cleansing is an essential step in the data preparation process and the Data Cleanse transformation makes it all the more convenient for users to cleanse raw data and present it in a more refined, standardized and enterprise-ready format. Using the Data Cleanse transformation, users can:
- Replace null strings and numeric values with blanks and zeros respectively
- Remove extra spaces, unnecessary text and numeric characters, punctuations, line breaks, etc.
- Change letter cases to upper, lower, or title case
- Apply regular expressions and find-and-replace actions on datasets
- Prepare raw data for transformation, validation, profiling, and record matching functions
Dynamic Layout¶
Dynamic Layout feature has been added to some source objects in Centerprise 8.0. It makes it all more convenient for users to ensure layout consistency in integration flows. The Dynamic Layout feature eliminates the need and hassle of manually updating the mappings between different objects in a dataflow and ensures they’re all in sync with the source schema.
Job Optimizer¶
Delivering on our promise of optimizing data integration processes for maximum agility and seamless performance, we have added the Job Optimizer functionality to the latest release of Centerprise 8.0. The Job Optimizer enables users to optimize ETL performance and minimize job execution time.
The Job Optimizer issues recommendations to optimize integration flows, at runtime in job trace. These recommendations can be removing repetitive sort operations on presorted data, apply an ORDER BY clause on database sources to run sort operations in the database, or any other suggestion to optimize your flow. As a result, it helps bring a significant increase in performance by efficiently managing the RAM, CPU, network and disk utilization while executing jobs.
The New Search Feature¶
A Search feature has been added to the Toolbox, Expression Builder, Server Log and Job Monitor. This new Search bar enables better navigation through different features and components, resulting in better experience and improved user-friendliness.
- In the Toolbox, the Search feature enables easy navigation across different sources, destinations, transformations, text processors, resources, and other components of the toolbox.
- In the Expression Builder, the Search features makes it convenient for users to find a relevant expression function without having to browse through the extensive library of Centerprise’s built-in expressions.
- In the Server Log, this feature allows users to look for particular jobs deployed, running, or executed on the Server with respect to JobID, Server Name, Component, Severity and Error Info.
- In the Job Monitor, the search feature allows users to look for jobs based on different criteria such as Date, Job Type, Job File, Server, etc.
Astera REST Web Service APIs¶
We have published an extensive library of Astera REST web service APIs. Now, you can perform a wide range of data integration functions by using APIs to directly communicate with the Centerprise server and get the required data. Here are the resources that are available for the users in the first round of the beta release:
- Account APIs: Resources related to account login and user authentication.
- Server APIs: Resources related to job monitoring, schedules, and server information.
User Roles and Access Control¶
Centerprise 8.0 comes with enhanced role-based security features, enabling users to manage data pipelines in heterogenous and multi-user environments. Administrators can grant or restrict access to various users within the organization, based on their role in the entire data management cycle. The three-fold security mechanism incorporated in Astera Centerprise 8.0 includes:
- Secure Domain Configuration – encrypts information exchange between Centerprise client and server through an SSL protocol
- User Authentication – prevents unauthorized access to data
- Role Management – enables administrators to centrally manage user access by creating custom roles and access controls
Centerprise 8.0 – What’s Improved¶
Server Monitoring and Job Management¶
Centerprise 8.0 includes advanced server monitoring and job management features to enable administrators to proactively manage job monitoring, prevent downtimes, and identify the causes of server failure. Moreover, in case of server failure or outage, users can generate a diagnostic file to get details about the state of the server and the machine where it’s deployed.
This diagnostic file contains information such as server status, server health, active connections, process memory size, thread count, installed drivers, etc. which you can use for troubleshooting and improvements. Furthermore, you can track, monitor and manage the jobs scheduled, running or executed on the server in the Job Monitor window.
Job Trace Improvements¶
The Job Trace window has been updated with an improved UI response time and better trace navigation. The trace refresh on the client is decoupled from the server writing to the trace. As a result, the client may not display all entries while loading a very fast and long live trace, which will ultimately improve the load time for long trace outputs. The server performance has also been optimized since the server uses bulk insert to write the trace output.
Here’s a rundown of the improvements in the job trace:
- Refresh and Clear Trace buttons have been added to the monitoring toolbar.
- Suggestions for job optimization are always displayed in the job trace.
- Verification errors or warnings or suggestions can be traced in the job trace.
- The status of the job is also displayed by an icon depending on whether it is completed, queued, terminated, etc.
Workflow Performance Optimization¶
Another significant highlight of Centerprise 8.0 is the improvement in Astera server code to optimize the performance, reliability, and scalability of workflows. The workflow execution logic has been optimized to eliminate any chance of a workflow hanging amidst session in any state (queued, running, or terminating), even if there are transient communication errors between the main server or cluster servers and the repository.
Significant improvements have been made to Resume and Terminate functionalities for workflows in Centerprise 8.0. The limitations in resuming complex, nested workflows found in previous releases have been resolved in Centerprise’s latest release.
In Centerprise 8.0, while working with nested workflows (workflows that call other workflows within a master workflow), if a child component returns an error and is not executed successfully, the parent flow will terminate automatically. Likewise, when you resume a child flow that returned error(s) and led to the termination of the parent flow, the parent flow will also get resumed automatically.
Parametrization¶
The parametrization functionality in Centerprise 8.0 has been improved to support expression compilation in variables. The syntax has been changed from ‘$ ()’ notation to ‘{}’. However, this update won’t affect the parameters incorporated in existing flows. The current flows will continue to run seamlessly as before, without having to manually update the syntax. And once you save any flow that was using previous notation ‘$ ()’ in version 8, it will automatically get converted to ‘{}’.
Other Highlights¶
Improvements in List Lookup and DB Lookup transformations: New options added to the layout builder screen of List lookup and DB lookup transformations. Now you can keep the source value for the values that do not match the replacing criteria in a list or a database table.
Path Mappings tab added: A new tab for specifying client and server path mappings has been added on the Cluster Settings This is particularly useful when the client and server reside on different networks or network file systems and their paths have to be mapped to correctly transfer files at run-time.
Option added to repair file paths: For better project management, we have added a new option to repair the file paths for files in any particular folder. This option is useful if you are working in a case-sensitive environment.
Shortcut for replacing parameters information for consolidated parameters added: Now you can replace the common parameter information for all objects in a flow with this command right from the dataflow screen.
Pushdown mode is expanded and improved: More transformations and expressions are now supported in the Pushdown mode. We have also improved the explanations for the verification errors for easier troubleshooting.
Salesforce UPSERT is fully supported: Salesforce UPSERT is now fully supported. UPSERT uses an external ID field in the Salesforce table to match incoming data with existing records.
Keyboard shortcuts added and displayed: Keyboard shortcuts are enabled for various task windows. You can view the shortcut keys next to their respective commands.
Performance improvements when mapping complex trees: Runtime performance of the dataflows containing complex tree structures is improved.
New additions to the data providers list: We have added Tableau to the list of supported data providers.
SCD write strategy improved: SCD type 3 and type 6 are now supported.
REST call retries: For the REST calls running in timeout error, Centerprise will automatically make up to three retry attempts to process the call.
Migrating from Centerprise 7.x to Centerprise 8.0¶
If you’re using an older version of Centerprise and planning to upgrade to Centerprise 8.0, follow through the steps mentioned in this document for a smooth migration.
Have any queries, issues or suggestions? Reach out to us at support@astera.com.