Manipulating Source Field Data

Manipulating Source Field Data

Shuttle allows users to manipulate Source data in an effort to improve data integrity. In order to accomplish this, Shuttle has leveraged the Nunjucks code library. Don't worry, you don't have to learn Nunjucks in order to use it. In this section we'll review a few examples of ways to transform your Source data. 
  1. Select the Source that you would like to edit.
  2. On the Mapping tab, Map the Source field you would like to manipulate to it's corresponding Relay field.

  3. Click on the Source field. From the Edit Source Field dialog box, you will be prompted to choose a Data type; Select Expression.
    1. Expression: Fields that you would like to enhance or manipulate before the data is mapped to the Relay Field. 
      1. Example: If you would like all values in the First Name field to have a capitalized first letter, you would use the field source code and the "capitalize" nunjuck code.
  4. Next, choose one of the following functions to manipulate the Source data: 
    • Filters: Use these functions to enhance the format of the data. 
      • DateEnhance the format of your date fields. (New Feature from Release 1.0.2)
        • Directions: Click into the Field value, and a pipe "|" and type one of the following:
          • date: Add this filter in order to have the field display in ISO format.
            Example: 2018-04-17T11:54:12-05:00
          • {{ pod.date | date }}
          • date (format): Add this filter in order to format the date field in the format you determine.
            Example: Tuesday, April 17th 2018, 11:54:12 am
          • {{ pod.date | date("dddd, MMMM Do YYYY, h:mm:ss a") }}
          • date (format, timezone): Add this filter in order to format the date field to a certain timezone.
            Example: Tuesday, April 17th 2018, 3:54:12 pm
          • {{ pod.date | date("dddd, MMMM Do YYYY, h:mm:ss a", "GMT") }}
      • Capitalize: Capitalizes the first letter in the field, and adjusts all other letters to be lowercase.
        • Directions: Click into the Field value, and add a pipe " | " and type "capitalize". 
        • Code Example: 
        • {{ pod.first_name | capitalize }}
        • Example: Raw data: john ; With Capitalize Filter: John
      • Lower: Lowercases all letters in the field.
        • Directions: Click into the Field value, and add a pipe " | " and type "lower".
        • Code Example: 
        • {{ pod.model_number | lower }}
        • Example: Raw data: MkGIll342 ; With Lower Filter: mkgill342
      • Round: Round numbers to the number of decimals indicated.
        • Directions: Click into the Field value, and add a pipe " | " , type "round", and type in the number of decimal points it should round to, "(2)".
        • Code Example: 
        • {{ pod.project_value | round(2) }}
        • Example: Raw data: $356.7895 ; With Round (2 decimal points) Filter: $356.79
      • Title: The first letter of every word in the sentence should be capitalized.
        • Directions: Click into the Field value, and add a pipe " | " and type "title".
        • Code Example: 
        • {{ pod.role | title }}
        • Example: Raw data: business Analyst ; With Title Filter: Business Analyst
      • Trim: Strip leading and trailing whitespace.
        • Directions: Click into the Field value, and add a pipe " | " and type "trim".
        • Code Example: 
        • {{ pod.project_type | trim }}
        • Example: Raw data: " Fountain  " ; With Trim Filter: "Fountain"
      • Truncate: Reduces the number of characters in a field. Characters are counted from left to right.
        • Directions: Click into the Field value, and add a pipe " | " , type "truncate", type how many characters from the left should appear (1), then type ,true,  "") to finish off the code.
        • Code Example: 
        • {{ pod.gender | truncate(1, true, "") }}
        • Example: Raw data: Female ; With Trim Filter: F
      • Upper: Capitalizes all letters in the field.
        • Directions: Click into the Field value, and add a pipe " | " and type "upper".
        • Code Example: 
        • {{ pod.company | upper }}
        • Example: Raw data: xyz company ; With Trim Filter: XYZ COMPANY
      • URL Encode: Escape strings for use in URLs, using UTF-8 encoding.
        • Directions: Click into the Field value, and add a pipe " | " and type "urlencode".
        • Code Example: 
        • {{ pod.url_source | urlencode }}
        • Example: Raw data: http://example.com/ ; With Trim Filter: http%3A%2F%2Fexample.com%2F
  5. Enter the function you would like to utilize into "Value" and click Change.

  6. The Expression has now been added to that field, and all PODs coming in through this Source will be enhanced using the included code.

There are many more functions available using Nunjucks not reviewed in this documentation because of the complex nature of the code. If you are interested in learning those functions, and how they could be used, reference this Nunjucks documentation. 

How did we do?