Challenge

Azure Stream Analytics exclusively supports functions written in .NET Standard 2.0. Handling JSON data often necessitates the utilization of tools such as Newtonsoft or features from System.Text.Json, both of which are NOT accessible in .NET Standard 2.0. Furthermore, another compelling reason to opt out for .NET is the complexity involved in storing and updating the compiled package to a designated path in Azure Blob Storage, followed by referencing it within the job configuration.

Loosing the comfort of C# and .net and switching to JavaScript wasn’t as hard as it seems to be in the beginning. The biggest challenge was to parse the incoming JSON correctly. Uppercase and lowercase characters can ruin your day.

JavaScript Calculations

The subsequent function provided is greatly simplified, merely adding two values to produce a new result. However, it’s important to note that you can perform highly intricate calculations as well.

function main(incomingData) {
    try {
        var result = calculateValues(incomingData);
        return result;
    }
    catch (err) {
        var result = {
            'newCalculatedValue': 0.0
        }
        return result;
    }
};

function calculateValues(incomingData) {
    var newCalculatedValue = incomingData.value1 + incomingData.value2;
    var result = {
        'newCalculatedValue': newCalculatedValue
    }
    return result;
}

I’ve implemented a catch block as a precautionary measure in case any “incorrect” values are received and cannot be converted accurately. Depending on the job’s settings, an uncaught exception could result in halting the job.

Calling the JavaScript Functions in the Stream Analytics Job Query

The query provided for the job is simplified to illustrate its usage.

WITH iothubstream AS
(
    SELECT 
        EventEnqueuedUtcTime,
        EventProcessedUtcTime,
        [IoTHub].ConnectionDeviceId AS ConnectionDeviceId,
        *
    FROM 
        inputiothub TIMESTAMP BY EventEnqueuedUtcTime
)
, calculateddata AS
(
    SELECT
        UDF.Calc(joineddata) as calculated,
        *
    FROM 
        iothubstream
)
, preparedView AS
(
    SELECT
        calculated.newCalculatedValue as newCalculatedValue,
        *
    FROM calculateddata
)

SELECT * 
INTO 
     outputblobstorage
FROM 
     reducedview

Conclusion

Creating custom values within a stream job using User-Defined Functions is straightforward in JavaScript. However, it’s not advisable to do so in the CLR (Common Language Runtime) way, as it only supports .NET Standard 2.0.

More Information