User Defined Transformers

Introduction

User defined Transfomers are a great way to configure a system transformer with your own presets and publish it everywhere. User defined transformers are saved at the account level and can be used across multiple jobs, saving you time during the schema configuration process. There are two ways to create User Defined Transformers:

1. Cloning System transformers
2. Creating a Custom Code Transformer

The following sections will walk through how you can create both types of User Defined Transformers.

Cloning System Transformers

Neosync ships with a number of system defined transformers that you can clone, update the configurations and then save it as a User Defined Transformer. This is helpful if you want to ensure that some transformers have certain configurations. For example, you may want to enforce that every column that uses a generate_int64 transformer only produces positive integers. Below we show how to clone a system transformer and use that transformer within a schema mapping.

Creating a User Defined Transformer

In order to create a user defined transformer, follow these steps:

1. Navigate to the Transformers page and click on + New Transformer.
2. You'll be brought to the new transformer page where you can select a base transformer. A base transformer serves as the blueprint for the user defined transformer. Select the base transformer for your user defined transformer.

3. Once you've selected a base transformer, you'll be prompted to give the transformer a name and description. Additionally, you can preset custom default configurations depending on the transformer. Fill out the details and click save.

Using a User Defined Transformer

1. Once you've created a user defined transformer, you'll see it appear in the transformer list in Transformers main page as well as the Schema configuration page. Above, we created a user defined transformer called custom-float-transfomer, we can now see it both places.

In the transformers table under the User Defined Transformer tab.

In the Schema configuration page in the transformer select.

Now we can finsih the rest of our job configuration and the newly created user defined transformer will be available in the transformer dropdown in the schema page.

Custom Code Transformers

Neosync also supports the ability to write your own custom logic in javascript for a transformer. We call this the transform_javascript transformer. Custom code tranformers take in an input value at the value keyword and execute your custom code against that value. Custom code tranformers are also only available for sync jobs since they require an input value.

Creating a Custom Code Transformer

In roder to create a custom code transformer:

1. Navigate to the Transformers page and click on + New Transformer.
2. You'll be brought to the new transformer page where you can select a base transformer. A base transformer serves as the blueprint for the user defined transformer. Select the Transform Javascript transformer.

3. Give your transformer a Name and a Description.
4. Then move onto to the Custom Code section. Here you can write in custom javascript code to transform your input value. Note that the value that you will be transforming is available at the value keyword and is of any type. For example, if the input value was john of type string then the following code:

let input = value + 'test';
return input;

You can also do more complicated transformations such as:

function manipulateString(str) {
  let result = reverseString(str);

  result = capitalizeString(result);

  return result;
}
// capitalie the string
function capitalizeString(s) {
  return s.toUpperCase();
}
// reverse the string
function reverseString(s) {
  return s.split('').reverse().join('');
}

const input = manipulateString(value);

return input;

Would return johntest. The code editor comes with autocomplete for most standard methods and syntax highlighting. Lastly, we do not currently support module imports in this section. Important: make sure you include a return statement or the custom function will not return a value.

5. Once you are satisified with your custom code, click on the Validate button to ensure that your javascript compiles and is valid. If it does not compile, we will return an invalid error.
6. Once your code has compiled, click on Submit to save your custom code transformer. You can now use your custom code transformer in sync jobs.