Validating JSON input in Node.js & TypeScript and cover these validation scenarios with Unit Tests
In the previous post I’ve created a sample backend with Node.js and TypeScript that can store comments via a POST call on /comments. I included some standard error handling for incorrect JSON syntax, but it didn’t actually check if the fields defined in our interface existed and contained correct values. Let’s add some validation for that now. Also, add unit tests for all error scenarios we have covered so far, so that we don’t accidentally break these when we alter our code even more.
Grabbing the template from last blog post
If you didn’t follow the previous blog post, make sure you have:
- A recent version of git
- Node.js and thus NPM installed
- IntelliJ IDEA or Webstorm. VSCode might also do the job but my project doesn’t contain settings for it.
- Postman or curl
On github, do a git clone https://github.com/Leejjon/node-ts-again and then switch to the 1.5 tag by running: git checkout tags/1.5
Verify that it works:
npm install- Open a new terminal and run
npm run webpack - Run
npm start - Run
curl -d '{"message": "Hi", "nickname":"Leejjon"}' -H "Content-Type: application/json" -X POST http://localhost:8080/comments; echo(the “; echo” is only needed in Linux to not mess up new lines in the terminal) and verify that you get a valid JSON response.
How to validate the fields in the request body?
In the previous post we already added JSON parsing which checks whether the request body contains valid JSON. But what happens if fields have an unexpected value or are missing? What happens if we just pass an empty object? Let’s try.
curl -d '{}' -H "Content-Type: application/json" -X POST http://localhost:8080/comments; echoRight now we don’t use the parameters yet, so nothing happens:
In the log statement we can see that it couldn’t grab the nickname parameter in the console.log statement in index.ts:
You might write some basic validation like this in the CommentsController.ts:
import {Request, Response} from "express";
import {CommentConfirmation, NewCommentRequest} from "../model/Comment";
export const postComment = async (req: Request, res: Response) => {
try {
const newCommentRequest: NewCommentRequest = req.body;
if (newCommentRequest.message && newCommentRequest.nickname) {
console.log(`Stored comment from ${newCommentRequest.nickname}.`);
let response: CommentConfirmation = {id: 'commentId', timestamp: new Date()};
res.status(200);
res.send(response);
} else {
res.status(400);
res.send("Invalid request.");
}
} catch (error) {
res.status(400);
res.send("Invalid request.");
}
}But then you might want to add a check if the strings aren’t empty or too long. You can add some regular expression / regex to verify this, you can see my quick attempt here:
const newCommentRequest: NewCommentRequest = req.body;
const {message, nickname} = newCommentRequest;
if (message && message.match(/^([a-zA-Z0-9_ .,:;-]){10,200}$/)
&& nickname && nickname.match(/^([a-zA-Z0-9-_]){3,20}$/)) {
console.log(`Stored comment from ${newCommentRequest.nickname}.`);
let response: CommentConfirmation = {id: 'commentId', timestamp: new Date()};
res.status(200);
res.send(response);
} else {
res.status(400);
res.send("Invalid request.");
}You can try it with curl:
curl -d '{"message": "This is a message, with a comma and dot.", "nickname": "Leejjon"}' -H "Content-Type: application/json" -X POST http://localhost:8080/comments; echoIf your request body grows with more fields, this if statements grows pretty big and becomes unreadable.
The class-validator library
An alternative to there big if statements with regular expression checks is to use the class-transformer and class-validator library.
You can install them with: npm install class-transformer class-validator
In the previous post, we created the Comments.ts file with the NewCommentRequest interface.
export interface NewCommentRequest {
message: string,
nickname: string
}
export interface CommentConfirmation {
id: string,
timestamp: Date
}We can convert this interface into a class with validation decorators on the fields they belong to. For the message I used the same regular expression, but for the nickname I could use a couple of predefined decorators that are much easier to read than a regular expression would be:
import {IsAlphanumeric, IsString, Length, Matches} from "class-validator";
export class NewCommentRequest {
@IsString()
@Matches(/^([a-zA-Z0-9_ .,:;-]){10,200}$/)
public message: string;
@IsString()
@Length(3, 20)
@IsAlphanumeric()
public nickname: string;
constructor(message: string, nickname: string) {
this.message = message;
this.nickname = nickname;
}
}
export interface CommentConfirmation {
id: string,
timestamp: Date
}Once you save this code in IntelliJ or WebStorm, you’ll get this error:
You need to enable experimental support for decorators in the tsconfig.json:
"experimentalDecorators": true,Start using the class-transformer and class-validator
See my implementation here in the CommentsController.ts:
import {Request, Response} from "express";
import {CommentConfirmation, NewCommentRequest} from "../model/Comment";
import {validate} from "class-validator";
import {plainToClass} from "class-transformer";
import {ValidationError} from "class-validator/types/validation/ValidationError";
export const postComment = async (req: Request, res: Response) => {
try {
let newCommentRequest: NewCommentRequest = plainToClass(NewCommentRequest, req.body as Object);
let validationErrors: Array<ValidationError> = await validate(newCommentRequest);
if (validationErrors.length > 0) {
validationErrors.forEach((validationError) => {
console.log(validationError);
});
res.status(400);
res.send("Invalid request.");
} else {
console.log(`Stored comment from ${JSON.stringify(newCommentRequest)}.`);
let response: CommentConfirmation = {id: 'commentId', timestamp: new Date()};
res.status(200);
res.send(response);
}
} catch (error) {
console.log(error);
res.status(400);
res.send("Invalid request.");
}
}If you now try to create a comment with a nickname that contains a non alphanumeric character like ‘@’, we’ll see that the validation library gives a nice descriptive error in the console log.
curl -d '{"message": "This is a message, with a comma and dot.", "nickname": "Leejjon@"}' -H "Content-Type: application/json" -X POST http://localhost:8080/comments; echo
It will also warn you when you leave fields out.
Update: Filtering out arrays
Something I found out long after writing this blog. While class-validator does perfect validation on any JSON objects with curly braces {} , whenever you pass an array with blocks [] in it:
curl -d '[]' -H "Content-Type: application/json" -X POST http://localhost:8080/comments; echo
You will see this is also valid JSON and the validator will completely ignore it. We can solve it for now simply by verifying whether the req.body starts and ends with curly braces.
import {Request, Response} from "express";
import {CommentConfirmation, NewCommentRequest} from "../model/Comment";
import {validate} from "class-validator";
import {plainToClass} from "class-transformer";
import {ValidationError} from "class-validator/types/validation/ValidationError";
export const postComment = async (req: Request, res: Response) => {
try {
if (!(req.body.startsWith('{') && req.body.endsWith('}'))) {
throw new Error('No curly braces found.');
}
let newCommentRequest: NewCommentRequest = plainToClass(NewCommentRequest, req.body as Object);
let validationErrors: Array<ValidationError> = await validate(newCommentRequest);
if (validationErrors.length > 0) {
validationErrors.forEach((validationError) => {
console.log(validationError);
});
res.status(400);
res.send("Invalid request.");
} else {
console.log(`Stored comment from ${JSON.stringify(newCommentRequest)}.`);
let response: CommentConfirmation = {id: 'commentId', timestamp: new Date()};
res.status(200);
res.send(response);
}
} catch (error) {
console.log(error);
res.status(400);
res.send("Invalid request.");
}
}Abstracting this validation stuff away from the business logic
We created a custom error handler for the JSON parser in the second blog post outside our bussiness logic (the postComment function in the CommentsController.ts). I don’t really want this validation logic in the postComment function. I want the postComment function to only focus on “posting a comment”!
But isn’t this validation part tied to the NewCommentRequest class that was created for the postComment function only? Well yes, but we can solve this by creating a generic middleware function in the index.ts. I took inspiration from the example from this blog post. Alter your index.ts with the following code:
..... other imports
import {NewCommentRequest} from "./model/Comment";... other code in index.tsfunction validationMiddleware<T>(type: any): RequestHandler {
return async (req, res, next) => {
// For some reason the class-transformer and class-validator don't see arrays as a validation error.
if (!(req.body.startsWith('{') && req.body.endsWith('}'))) {
res.status(400);
res.send("Invalid request.");
}
let validationErrors = await validate(plainToClass(type, req.body));
if (validationErrors.length > 0) {
validationErrors.forEach((validationError) => {
console.log(validationError);
});
res.status(400);
res.send("Invalid request.");
} else {
next();
}
};
}
app.post("/comments", validationMiddleware(NewCommentRequest), postComment);
You can clean up the CommentsController.ts after this quite a bit:
import {Request, Response} from "express";
import {CommentConfirmation, NewCommentRequest} from "../model/Comment";
import {plainToClass} from "class-transformer";
export const postComment = async (req: Request, res: Response) => {
try {
let newCommentRequest: NewCommentRequest = plainToClass(NewCommentRequest, req.body as Object);
console.log(`Stored comment from ${JSON.stringify(newCommentRequest)}.`);
let response: CommentConfirmation = {id: 'commentId', timestamp: new Date()};
res.status(200);
res.send(response);
} catch (error) {
console.log(error);
res.status(400);
res.send("Invalid request.");
}
}Writing unit tests that cover the error scenarios
You’re not really going to test manually with the curl commands that invalid JSON or JSON with missing fields are still handled correctly every time you change code and update your npm libraries. I am going to show how to set up simple tests for this without explaining what every library does.
Install the following dependencies:
npm i mocha chai sinon supertest ts-node @types/mocha @types/chai @types/sinon @types/supertest --save-dev
First add an export statement in the index.ts so we can easily use the server:
export const server = app.listen(PORT, () => {
console.log(`Listening on port ${PORT}`);
});In the package.json you can add a test script:
"scripts": {
"start": "node dist/index",
"webpack": "webpack --config webpackdev.config.ts",
"build": "webpack --mode production",
"test": "mocha -r ts-node/register src/**/*test.ts"
},Create a test file called CommentsController.test.ts in the same folder as the CommentsController.ts. I created one test for a successful call:
import 'mocha';
import {expect} from 'chai';
import {server} from "../index";
import {agent as request} from 'supertest';
describe("Test the API calls", () => {
after((done) => {
server.close(done);
});
it('Test the happy flow', async () => {
const res = await request(server)
.post('/comments').send({
message: "This is a post.,",
nickname: "Leejjon"
});
expect(res.status).to.equal(200);
expect(res.body.id).to.equal("commentId");
});
});This test approach uses one server for all tests within a describe block. This only works if your server is stateless. If it starts to contain state (in memory or an external database) you have to make sure your read the documentation of Mocha on how to start the server before every test and shut it down after every test to avoid tests affecting each others results.
You can run the tests with npm run test Or in IntelliJ rightclick on the test in the file:
Let’s add another test with incorrect JSON syntax:
import 'mocha';
import {expect} from 'chai';
import {server} from "../index";
import {agent as request} from 'supertest';
describe("Test the API calls", () => {
after((done) => {
server.close(done);
});
it('Test the happy flow', async () => {
const res = await request(server)
.post('/comments').send({
message: "This is a post.,",
nickname: "Leejjon"
});
expect(res.status).to.equal(200);
expect(res.body.id).to.equal("commentId");
});
it('Send invalid json syntax', async () => {
const res = await request(server)
.post('/comments').type('json').send('{"invalid"}');
expect(res.status).to.equal(400);
});
});And one with a valid json request that is missing the nickname field, just to verify that our validation with decorators works:
import 'mocha';
import {expect} from 'chai';
import {server} from "../index";
import {agent as request} from 'supertest';
describe("Test the API calls", () => {
after((done) => {
server.close(done);
});
it('Test the happy flow', async () => {
const res = await request(server)
.post('/comments').send({
message: "This is a post.,",
nickname: "Leejjon"
});
expect(res.status).to.equal(200);
expect(res.body.id).to.equal("commentId");
});
it('Send invalid json syntax', async () => {
const res = await request(server)
.post('/comments').type('json').send('{"invalid"}');
expect(res.status).to.equal(400);
});
it('Send valid json with missing nickname field', async () => {
const res = await request(server)
.post('/comments').send({
message: "This is a post.,"
});
expect(res.status).to.equal(400);
});
});You can now run all tests together with npm run test or launch it from IntelliJ/WebStorm:
That’s it for this post. You can download the full code from GitHub.
