Azure API Management validation

I had the opportunity to look at the new validation functionality for APIM. I summarize my thoughts to remember and share with others. You find the documentation post here: https://docs.microsoft.com/en-us/azure/api-management/validation-policies

We have four ways to validate requests and responses, content, parameters, headers, and status code. Three actions can be taken, ignore, detect, or prevent. Ignore will skip the validation, detect will log the validation error but not interrupt execution and prevent will stop processing on first error. Validations have a high-level settings that tells what to do with a specified or unspecified settings.

Note: As stated in the documentation I needed to reimport my API using management API version 2021-01-01-preview or later. I did this with a PowerShell script that you can find here APIM-Examples/Validation at main · skastberg/APIM-Examples (github.com)

Content

Validates the size of a request or response, also ensure that the body coming in or out follow the specification in the API description. For schema validation we’re limited to json payload. Content-Type validation is checked towards the API specification. Size validation works with all content types up to 100KiB payload size. This validation can be applied on all scopes for sections inbound, outbound and on-error.

The policy name is validate-content, details on usage see here: https://docs.microsoft.com/en-us/azure/api-management/validation-policies#validate-content

Content validation examples

To ensure that the specified content-type is honoured, set unspecified-content-type-action to prevent and to limit the size of a request set max-size and size-exceeded-action to prevent.

<policies>
    <inbound>
        <base />
        <validate-content unspecified-content-type-action="prevent" max-size="102400" size-exceeded-action="prevent" errors-variable-name="requestBodyValidation" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

APIM will return a status code 400 for any request with a body that exceeds the max-size.

Resulting Status code 400 - max size exceeded
Resulting Status code 400 – max size exceeded

If the request doesn’t set the correct content-type a status code 400 is returned. In this case the required type is application/xml but the provided is text/xml.

Resulting Status Code 400 - unspecified content type
Resulting Status Code 400 – unspecified content type

Preventions will generate exceptions that can be seen in Application Insights, also you could Trace the errors. The image shows a query in Application Insights showing the exceptions joined with the request information.

Preventions will generate exceptions, here seen in Application Insights
Preventions will generate exceptions, here seen in Application Insights

In this example the policy validates size and content-type as the previous one and in addition the content element specifies to validate the payload.  

<policies>
    <inbound>
        <base />
        <validate-content unspecified-content-type-action="prevent" max-size="102400" size-exceeded-action="prevent" errors-variable-name="requestBodyValidation">
            <content type="application/json" validate-as="json" action="prevent" />
        </validate-content>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>|

If the payload doesn’t conform the json schema a status code 400 is returned with at descriptive message.

Resulting Status Code 400 -  Invalid json
Resulting Status Code 400 – Invalid json

Parameters

Apart from the content(body) we receive data as parameters, header, query, or path. This validation checks the incoming parameters against the API specification. Each parameter type has its own element in the policy, depending on needs one or more are used. The API specification will show how parameters are expected, types and if mandatory or not. This validation can be applied on all scopes for the inbound section.

The policy name is validate-parameters, details on usage see here: https://docs.microsoft.com/en-us/azure/api-management/validation-policies#validate-parameters

Parameter validation examples

Any parameters

With the following policy with unspecified-parameter-action set to prevent, any parameter of any kind in the request that is not in the API specification will be stopped.

<policies>
    <inbound>
        <base />
        <validate-parameters specified-parameter-action="ignore" unspecified-parameter-action="prevent" errors-variable-name="requestParametersValidation" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

The resulting status code 400 response from APIM. In this case a header but it could be something else depending on your specification.

Resulting Status Code 400 -  Unspecified header
Resulting Status Code 400 – Unspecified header

Path

In this example an operation has the required parameter, format, that we want to validate before sending to the backend. If the request with wrong type for format is received, we get a 400 error from APIM. As we set specified-parameter-action and unspecified-parameter-action to ignore other errors will be disregarded.

The required integer parameter in APIM
The required integer parameter in APIM
<policies>
    <inbound>
        <base />
        <validate-parameters specified-parameter-action="ignore" unspecified-parameter-action="ignore" errors-variable-name="requestParametersValidation">
            <path specified-parameter-action="detect">
                <parameter name="format" action="prevent" />
            </path>
        </validate-parameters>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

In this case the response is more generic not specifying the incorrect parameter, that said the exception seen in Application Insights is more specific.

Resulting Status Code 400 -  with a generic error message
Resulting Status Code 400 – with a generic error message

Headers

In this example an operation has the required header, spoken-language, that we want to validate before sending to the backend. The header is an enumeration, and we want to validate that the correct values are used before sending to the backend.

The required header in APIM
The required header in APIM

To prevent that the request is forwarded if an invalid value in the spoken-language header we can use the following policy.

<policies>
    <inbound>
        <base />
        <validate-parameters specified-parameter-action="prevent" unspecified-parameter-action="ignore" errors-variable-name="requestParametersValidation">
            <headers specified-parameter-action="detect" unspecified-parameter-action="ignore">
                <parameter name="spoken-language" action="prevent" />
            </headers>
        </validate-parameters>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

A request with an invalid value returns status code 400.

Returned Status Code 400 telling the value is invalid for spoken-language
Returned Status Code 400 telling the value is invalid for spoken-language

Query

In this example an operation has the query parameter, dayno, that we want to validate before sending to the backend. The header is an integer, and we want to validate that the correct type is used before sending to the backend.

<policies>
    <inbound>
        <base />
        <validate-parameters specified-parameter-action="ignore" unspecified-parameter-action="ignore" errors-variable-name="requestParametersValidation">
            <query specified-parameter-action="detect" unspecified-parameter-action="ignore">
                <parameter name="dayno" action="prevent" />
            </query>
        </validate-parameters>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

A request with a non integer value will be prevented and a status code 400 returned by APIM.

 Returned Status Code 400 telling the request could not be processed
Returned Status Code 400 telling the request could not be processed

Headers

Just as we validate incoming parameters it might be necessary to validate that our outbound data adheres to the API specification. This validation checks that the response headers is of the type we have specified in the API description. This validation can be applied on all scopes for sections outbound and on-error.

The policy name is validate-headers, details on usage see here: https://docs.microsoft.com/en-us/azure/api-management/validation-policies#validate-headers

Example

In this example an operation that has the Test-Header specified as integer.

Header in the specification
Description of Test-Header

In this policy we detect specified headers but don’t act on any errors except Test-Header. Unspecified headers will be ignored.

<policies>
    <inbound>
        <base />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
        <validate-headers specified-header-action="detect" unspecified-header-action="ignore" errors-variable-name="responseHeadersValidation">
            <header name="Test-Header" action="prevent" />
        </validate-headers>
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

Returning a response of wrong type will result in a status code 400 with a generic text “The request could not be processed due to an internal error. Contact the API owner.” if you look into Application Insights the message is more clear “header Test-Header cannot be validated. Input string ‘yourvalue’ is not a valid number. Path ”, line 1, position 3.

Generic response with status code 400.
Generic response with Status Code 400

Status code

An important part of our API specifications are the status codes we return. This validation check the HTTP status codes in responses against the API schema. This validation can be applied on all scopes for sections outbound and on-error.

The policy name is validate-status-code, details on usage see here: https://docs.microsoft.com/en-us/azure/api-management/validation-policies#validate-status-code

Example

In this example an operation that has only status code 200 specified. If the requested session doesn’t exist the backend will return Status Code 404. We want to validate all not specified status codes except 404.

Frontend with status code 200 response specification
Only Status Code 200 is specified

To avoid the status code 502 that will be the result if we validate any unspecified status codes we add a status-code element with action set to ignore.

<policies>
    <inbound>
        <base />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
        <validate-status-code unspecified-status-code-action="prevent" errors-variable-name="variable name">
            <status-code code="404" action="ignore" />
        </validate-status-code>
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

The prevented response if we don’t ignore the 404 status code.

Status 502 if an unspecified header is returned
Status 502 if an unspecified header is returned

The exception in Application Insights

Summary

This is definitely a set of policies that we can use to ensure that a API specification is honored. It will require some thinking to balance the trade off between the added value and the performance implication of doing the validation.

15 Comments

  1. Hi! We are trying to validate JSON properties in the inbound processing in API Management. We have created the schema with the required fields in a definition and put that as the request definition. This is the definition called Contact:

    {
    “type”: “object”,
    “properties”: {
    “name”: {
    “type”: “string”
    },
    “email”: {
    “type”: “string”
    }
    },
    “required”: [
    “name”, “email”
    ]
    }

    Then we have added to the inbound processing:

    The request still goes through, however we would like to get the behavior as in your example. Any suggestions or ideas are very much welcomed!

    Have a nice weekend!

    Reply

    1. Hi!
      I did a quick check with missing required element and it works as expected. The response I get is

      {
      “statusCode”: 400,
      “message”: “Body of the request does not conform to the definition ApiSkiResortPostRequest, which is associated with the content type application/json. Required properties are missing from object: name. Line: 6, Position: 3”
      }

      Are you specifying content type (application/json)? Did you recreate the API with the right API version?
      Have a nice weekend!

      Reply

      1. Yes, we have put this inside of the validate-content tag : content type=”application/json” validate-as=”json” action=”prevent”. Is that correct?

        What do you mean by using the right api version? We have created a new revision of our API inside our API Management, but that still gives the same result (the request goes through, however we want to prevent).

      2. Yes, it looks right. You have a note early in the post about reimporting the API using management API version 2021-01-01-preview or later. If you don’t do that as per today most of the validations won’t work, don’t know exactly which ones as I did it to start with. You have a link to my GitHub repo where you find a PowerShell script that does it for the Conference API, you just need to adapt it to your API.

  2. Where is the JSON schema been stored for validation, Like I have a custom JSON with Custom fields which we want to be validated using apim policy only. Is there a way to achieve this.

    Reply

  3. Hi Samuel,

    Thanks so much for the above information…really really useful!!!

    I have implemented the above and managed to get policy to work partially against the JSON schema but it is not validating against the whole schema. It is only validating against data types and required fields but I can add any field to the JSON payload and even though the field is not in the JSON schema it will not fail which.

    Any ideas why this is happening?

    I know you mentioned reimporting the API using management API version 2021-01-01-preview or later…is this still required?

    Reply

    1. Hi Nahim,

      Thanks, happy it helps!

      As you see in the post I did a test providing a name that does not exist in the specification so it should work. I assume but not sure, that if you imported with a prior version of the API you still need to reimport.

      Reply

  4. Hi Samuel,

    thanks for this article. I found it because I am trying to do the following…
    The POST-request has a parameter called ‘signature’. This is a HMACSHA256-encoding of the raw request body with a key.

    So as an inbound policy I want to check whether the given signature is identical to my own encoding…
    I succeeded in calculating the hash.

    My question: can I use validate-parameter to test whether they are equal and prevent execution if not?

    Thanks

    Reply

    1. Hi Wouter,
      Not to my knowledge, I have not seen that you can do that kind of thing. I would probably set a variable with the calculation and compare it choosing to forward or fail depending on the outcome. That said, I would be very careful as it can have a performance hit on the entire environment.

      Reply

  5. HI API- All Description must be valid how to check please provide some example scripts

    Reply

    1. Hi,

      Can you please clarify what you are looking for?
      There is no check everything, you use the validations that are important for your setup. Remember that each validation add processing overhead.

      Thanks
      Samuel

      Reply

Leave a comment