Assette Functions Table Helper Data Block

The Assette Functions Table Helper (ast_fn_table_helpers) Data Block is managed by Assette and is not editable. This block is intended for use as a utility library. It should not be used as a standalone Data Block.

The ast_fn_table_helpers (ast_fn_table_helpers) Data Block in Assette is a Client Python Library used to support data manipulation and formatting operations across other Data Blocks, particularly transformation and interface blocks. It encapsulates a set of helper functions that handle common tasks such as filtering dictionaries, cleaning invalid values, formatting dates, and transforming data structures. These utilities standardize and simplify downstream logic in reusable and readable ways, making them essential for clean, client-ready outputs.

General Info #

Field	Value
Name	ast_fn_table_helpers
Block Category	Configuration
Block Type	Client Python Library
Data Category	None
Output Type	Data Table
Editable	False

Dependencies #

Name	Dependent Block
ast_table_helpers_env	ast_table_helpers_env [Link]

Helper Function Reference and Usage #

The following utility functions are available in the Assette Functions Table Helper (ast_fn_table_helpers) Data Block.

Function Name	Purpose	Example Usage
FilterDictionaryByFields	Filters a list of dictionaries based on matching key-value pairs.	FilterDictionaryByFields(data, {"Code": "12345"})
clean_nan	Replaces NaN values with None for all or selected fields in a list of dictionaries.	clean_nan(data) or clean_nan(data, ["Revenue", "NAV"])
format_date	Converts date strings or objects to standardized string formats using predefined or custom formats.	format_date("2024-06-30", "mm/dd/yyyy")
transform_dictionary_list	Renames dictionary keys in a list of dictionaries based on a mapping dictionary.	transform_dictionary_list(data, {"AccountNumber": "Code"})

Example Usage #

The following is a basic example of how this helper block might be used within a parent Data Block.

Output = read("OrganizationReadLocal",{})
# Standardize field names
Output = ast_fn_table_helpers.transform_dictionary_list(Output.get("data"), {"AccountNumber":"Code"})
# Filter by provided parameters
Output = ast_fn_table_helpers.FilterDictionaryByFields(Output, {"Code": "12345"})
# Clean up invalid values
Output = ast_fn_table_helpers.clean_nan(Output)
response["data"] = Output

Clean NaN Function #

The clean_nan function replaces all occurrences of NaN (Not a Number) values with None in a list of dictionaries (often representing a data table). This is useful when preparing data for serialization (e.g., converting to JSON) or when NaN values are not handled properly by downstream processes. It is suggested to clean specific fields that are either known to contain NaN or otherwise limit the scope of clean_nan in order to maintain reasonable loading times.

• NaN Detection: The helper function is_nan checks whether a value is a floating-point NaN. It uses math.isnan(value) after ensuring the value is a float.
• Mutable Default Arguments: The fields parameter defaults to None instead of an empty list to avoid issues with mutable default arguments.
• In-place Modification: The function modifies the dictionaries in the data_table list in place.
• Data Integrity: The function assumes that the values associated with the keys in the dictionaries can be of any type. It only replaces values that are NaN floats.

Syntax #

ast_fn_table_helpers.clean_nan(data_table, fields=None)

Parameters #

• data_table (list of dict): The data to clean. It’s a list where each element is a dictionary representing a record with key-value pairs.
• fields (list of str, optional): Specific keys in the dictionaries to check for NaN values. If not provided or if it’s an empty list, the function will check all keys in each dictionary.

Returns #

• list of dict: The cleaned data_table, where all NaN values have been replaced with None.

Data Types #

• data_table: list[dict[str, any]]
• fields: list[str] or None
• Returns: list[dict[str, any]]

Examples #

Cleaning All Fields #

data = [
    {'a': 1.0, 'b': 'nan', 'c': 3.0},
    {'a': 'nan', 'b': 2.0, 'c': 'nan'},
    {'a': 4.0, 'b': 5.0, 'c': 6.0}
]

cleaned_data = ast_fn_table_helpers.clean_nan(data)

# Output:
# [{'a': 1.0, 'b': None, 'c': 3.0}, {'a': None, 'b': 2.0, 'c': None}, {'a': 4.0, 'b': 5.0, 'c': 6.0}]

Cleaning Specific Fields #

data = [
    {'a': 1.0, 'b': 'nan', 'c': 3.0},
    {'a': 'nan', 'b': 2.0, 'c': 'nan'},
    {'a': 4.0, 'b': 5.0, 'c': 6.0}
]

cleaned_data = ast_fn_table_helpers.clean_nan(data, fields=['a', 'c'])
print(cleaned_data)
# Output:
# [{'a': 1.0, 'b': math.nan, 'c': 3.0}, {'a': None, 'b': 2.0, 'c': None}, {'a': 4.0, 'b': 5.0, 'c': 6.0}]

Format Date Function #

The format_date function formats a date input into a specified string format. It accepts a date input, which can be a datetime.datetime object, a datetime.date object, or a date string. It then formats this input according to a predefined format or a custom format provided. Note that if the preferred format is not available in the list, users can use Python’s date formatting by specifying the format string directly.

Syntax #

ast_fn_table_helpers.format_date(input, format='iso')
#input: The date input to format. Can be a datetime.datetime, datetime.date, or str.
#format: The format to output the date in. Defaults to 'iso'.

Data Types #

• input: datetime.datetime, datetime.date, or str
• format: str
• Returns: str

Parameters #

• input (datetime.datetime, datetime.date, or str): The date to be formatted.
• format (str, optional): The desired output format. Defaults to 'iso'.

Predefined Formats #

You can use any of the following predefined format keywords for the format parameter , or alternatively, users can provide a custom format string compatible with strftime.

• 'iso': 'YYYY-MM-DD' # ISO 8601 date (default)
• 'iso_datetime': 'YYYY-MM-DDTHH:MM:SS' # ISO 8601 datetime
• 'yyyymmdd': 'YYYYMMDD'
• 'yyyyddmm': 'YYYYDDMM'
• 'y-m-d': 'Y-M-D'
• 'd-m-y': 'D-M-Y'
• 'm-d-y': 'M-D-Y'
• 'ddmmyyyy': 'DDMMYYYY'
• 'mmddyyyy': 'MMDDYYYY'
• 'dd-mm-yyyy': 'DD-MM-YYYY'
• 'mm-dd-yyyy': 'MM-DD-YYYY'
• 'dd.mm.yyyy': 'DD.MM.YYYY'
• 'mm.dd.yyyy': 'MM.DD.YYYY'
• 'yyyy/mm/dd': 'YYYY/MM/DD'
• 'dd/mm/yyyy': 'DD/MM/YYYY'
• 'mm/dd/yyyy': 'MM/DD/YYYY'
• 'month_day_year': 'Month DD, YYYY' Full month name, day, year (e.g., December 21, 2023)
• 'year_month_day_time': 'YYYY-MM-DD HH:MM:SS' # Date and time

Examples #

Formatting to ISO Datetime String #

Covert a date in the YYYY-MM-DD format to ISO Datetime. This is the default format.

formatted_date = ast_fn_table_helpers.format_date('2023-10-05T14:30:00', 'iso_datetime')
# Output: '2023-10-05T14:30:00'

Formatting to ISO DD-MM-YYYY #

Covert a date in the YYYY-MM-DD format to DD-MM-YYYY.

formatted_date = ast_fn_table_helpers.format_date('2024-11-07', 'dd-mm-yyyy')
# formatted_date will be '07-11-2024'

Formatting to ISO DD-MM-YYYY #

Convert a date in the YYYY-MM-DD format to the full month name, day, and year.

formatted_date= ast_fn_table_helpers.format_date("2024-05-25", "%B %d, %Y")
# formatted_date will be ‘May 25, 2024’

Formatting with a Custom Format #

Convert a date in the YYYY-MM-DD format to a customized format.

formatted_date = ast_fn_table_helpers.format_date('2023-10-05', '%A, %B %d, %Y')

# Output: 'Thursday, October 05, 2023'

Filter Dictionary by Fields Function #

The FilterDictionaryByFields function filters a list of dictionaries based on specified field values provided in a filter dictionary. It checks each dictionary in the input list and includes it in the result if it meets all the criteria specified. Note that the FilterDictionaryByFields function assumes that the values in FilterFieldsDictionary are strings containing comma-separated values. Additionally, the comparison is performed by converting both the field values and the filter values to strings. If a key in FilterFieldsDictionary is not present in a dictionary from InputDictionaryList, that dictionary will not be included in the result. The function does not handle type conversion beyond string conversion using str().

Syntax #

ast_fn_table_helpers.FilterDictionaryByFields(InputDictionaryList, FilterFieldsDictionary)

Parameters #

• InputDictionaryList (list of dict): The list of dictionaries to be filtered.
• FilterFieldsDictionary (dict): A dictionary where each key is a field name, and each value is a string of acceptable values for that field, separated by commas.

Returns #

• list of dict: A list containing dictionaries that match all the filtering criteria.

Data Types #

• InputDictionaryList: list[dict]
• FilterFieldsDictionary: dict[str, str]

Examples #

Filtering by Single Field #

input_list = [
    {'id': 1, 'status': 'active', 'category': 'A'},
    {'id': 2, 'status': 'inactive', 'category': 'B'},
    {'id': 3, 'status': 'active', 'category': 'C'}
]

filter_fields = {'status': 'active'}

result = ast_fn_table_helpers.FilterDictionaryByFields(input_list, filter_fields)

# Output: [{'id': 1, 'status': 'active', 'category': 'A'}, {'id': 3, 'status': 'active', 'category': 'C'}]

Filtering by Multiple Fields #

input_list = [
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'Los Angeles'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'},
    {'name': 'David', 'age': 30, 'city': 'Chicago'}
]

filter_fields = {'age': '30,35', 'city': 'Chicago'}

result = ast_fn_table_helpers.FilterDictionaryByFields(input_list, filter_fields)

# Output: [{'name': 'Charlie', 'age': 35, 'city': 'Chicago'}, {'name': 'David', 'age': 30, 'city': 'Chicago'}]

No Matches Found #

input_list = [
    {'product': 'Laptop', 'price': 1000},
    {'product': 'Tablet', 'price': 500},
    {'product': 'Smartphone', 'price': 800}
]

filter_fields = {'price': '1500'}

result = ast_fn_table_helpers.FilterDictionaryByFields(input_list, filter_fields)

# Output: []

Transform Dictionary List Function #

The transform_dictionary_list function modifies the keys of dictionaries within a list based on a provided key mapping. It returns a new list of dictionaries with the keys transformed according to the mapping specified in transform_detail_dict.

• Unchanged Keys: If a key in the dictionaries is not present in transform_detail_dict, it remains unchanged.
• Missing Keys: The function does not add or remove keys from the dictionaries; it only changes existing keys.
• Immutable Input: The original dict_list is not modified; the function returns a new list with transformed dictionaries.
• Data Integrity: Values associated with keys remain unchanged; only the keys are transformed.

Syntax #

ast_fn_table_helpers.transform_dictionary_list(dict_list, transform_detail_dict)

Parameters #

• dict_list (list of dict): The list of dictionaries to be transformed.
  • Each dictionary in the list represents an item with key-value pairs.
• transform_detail_dict (dict): A dictionary that maps original keys to new keys.
  • Keys: Original keys present in the dictionaries of dict_list.
  • Values: New keys to replace the original keys.

Returns #

• list of dict: A new list of dictionaries with keys transformed according to transform_detail_dict.

Data Types #

• dict_list: list[dict]
• transform_detail_dict: dict[str, str]
• Returns: list[dict]

Examples #

Basic Key Transformation #

dict_list = [
    {'name': 'Alice', 'age': 30},
    {'name': 'Bob', 'age': 25}
]

transform_detail_dict = {'name': 'full_name', 'age': 'years_old'}

transformed_list = ast_fn_table_helpers.transform_dictionary_list(dict_list, transform_detail_dict)

# Output: [{'full_name': 'Alice', 'years_old': 30}, {'full_name': 'Bob', 'years_old': 25}]

Partial Key Transformation #

dict_list = [
    {'id': 1, 'value': 100},
    {'id': 2, 'value': 200}
]

transform_detail_dict = {'value': 'amount'}

transformed_list = ast_fn_table_helpers.transform_dictionary_list(dict_list, transform_detail_dict)

# Output: [{'id': 1, 'amount': 100}, {'id': 2, 'amount': 200}]

No Key Transformation #

dict_list = [
    {'id': 1, 'value': 100},
    {'id': 2, 'value': 200}
]

transform_detail_dict = {}

transformed_list = ast_fn_table_helpers.transform_dictionary_list(dict_list, transform_detail_dict)

# Output: [{'id': 1, 'value': 100}, {'id': 2, 'value': 200}]