TDD for Express and MySql. Simple Contact Us form
A simple Express/Mysql Contact Us app is a good fit to discuss what a production ready, testable and scalable Express project structure could be. Our goal is to populate MySql table with user messages and personal details. The project is intentionally simple, to illustrate essential points, but it is, hopefully, representative enough to provide a useful initial template for real life applications. Source code is available on Github. It includes both server side Express application with MySql persistance, and a simple Contact Us form. When in development, the project runs server and MySql database locally, using correspondingly pm2 manager and docker. To learn more about database schema, see blog post about migration scripts.
Express application structure
For scalability and better testability, when planing a new project, it is better to stick with best practices. So I usually use the following (pretty common) directory layout:
- models - files in this directory are responsible for interaction with database(s) (in our case MySql)
- controllers - to handle HTTP requests, to validate incoming requests, handle business logic using services (no need here because of the simplicity of the project) and persist data using models
- routes - to configure API endpoints
- scripts - various maintenance tasks, in our case seeding database with the help of faker
- sql/migration - directory for
*.sqlfiles to modify database schema or to import some static data. Also contains Dockerfile to build Docker container with up-to-date schema for development - test - for jest test files
As I already mentioned, initially our application may wish just to be able to add new user messages and return all messages available. Using this simple requirements, lets illustrate the code, one may typically find in each directory.
Model
Models implement all database related operations (retrieval, insertion, various searches). All model modules also reuse file db.js which opens database connections. In our case, initially, directory also contains contact-us.model.js with minimum necessary functions:
const sql = require("./db")
const ContactUs = {}
ContactUs.create = (newContact) => {
return new Promise((resolve, reject) => {
sql.query("INSERT INTO CONTACT_US SET ?", newContact, (err, res) => {
if (err) {
console.log("error: ", err)
reject(err)
return
}
console.log("created contact-us: ", { id: res.insertId, ...newContact })
resolve({ id: res.insertId, ...newContact })
})
})
}
ContactUs.findById = (contactId) => {
return new Promise((resolve, reject) => {
sql.query(`SELECT * FROM CONTACT_US WHERE id = ${contactId}`, (err, res) => {
if (err) {
console.log("error: ", err)
reject(err)
return
}
if (res.length) {
console.log("found contact-us: ", res[0])
resolve(res[0])
return
}
// not found ContactUs with the id
resolve(null)
})
})
}
ContactUs.removeAll = () => {
return new Promise((resolve, reject) => {
sql.query("DELETE FROM CONTACT_US", (err, res) => {
if (err) {
console.log("error: ", err)
reject(err)
return
}
console.log(`deleted ${res.affectedRows} contact-us`)
resolve(res)
})
})
}
ContactUs.getAll = () => {
return new Promise((resolve, reject) => {
sql.query("SELECT * FROM CONTACT_US ORDER BY ID", (err, res) => {
if (err) {
console.log("error: ", err)
reject(err)
return
}
console.log("contact-us: ", res)
resolve(res)
})
})
}
module.exports = ContactUs
Controllers
Our contact-us.controller.js module is responsible for simplest request validation/error handling:
const Contact = require("../models/contact-us.model")
// Create and Save a new Message
exports.create = async(req, res) => {
// Validate request
// Later more sanity checking could be added here
if (!req.body) {
res.status(400).send({
message: "Content can not be empty!"
})
}
try {
await Contact.create(req.body)
res.send({message: "Success!"})
} catch (err) {
console.error(err)
res.status(500).send({
message: err.message || "Some error occurred while saving your message."
})
}
}
// Retrieve all Messages from the database.
exports.findAll = async(_req, res) => {
try {
const contacts = await Contact.getAll()
res.send(contacts)
} catch (err) {
console.error(err)
res.status(500).send({
message: err.message || "Some error occurred while retrieving messages."
})
}
}
Routes
Files here work essentially as configuration files, responsible to configure our API endpoints:
module.exports = app => {
const contacts = require("../controllers/contact-us.controller")
// Create a new Contact
app.post("/contacts", contacts.create)
// Retrieve all Contacts
app.get("/contacts", contacts.findAll)
}
Server
Our server.js file wires it all together (adding a few endpoints for easy app diagnostic):
const express = require('express')
const bodyParser = require('body-parser')
const path = require('path')
const cors = require('cors')
const PORT = process.env.PORT || 8080
const app = express()
//STATIC FOLDER
app.use(express.static(path.join(__dirname, '../client/build')))
// Body Parser Middleware
app.use(bodyParser.json())
// Deal with CORS
app.use(cors())
// Start Express listening
app.listen(PORT, () => {
console.log(`Server started on port ${PORT}`)
})
// Test to make sure the API can talk to React
app.get('/', (_req, res) => {
return res.json({
message: 'API Active'
})
})
// An echo, to help with debugging
app.post('/', (req, res) => {
return res.json({
message: 'ECHO!',
posted: req.body
})
})
require("./routes/contact-us.routes")(app)
TDD using Jest. Easy MySql mocking
Such complicated project structure may look an overkill for such a simple project, but we will rip benefits of our approach when extending and maintaining our project later. Even now though, such project structure helps me to test code easily. Also, with such a setup, a lot of application functionality could be tested without real database (all functionality, except models). Developers' time is a precious (and pricey) asset, and running tests quick could be a big win.
First, in every application, database quires need to be tested on real database. As all database related functionality resides in models, I can easily test my model functions against development MySql database running in a Docker container:
describe('model methods', () => {
it('contact is created and queried back', async () => {
const req = {
first_name: 'name',
email: 'email@email.com',
message: 'message',
last_name: 'surname',
phone: 'phone',
address: 'address'
}
const contact = require("../../models/contact-us.model")
const res1 = await contact.create(req)
expect(res1).toMatchObject(req)
const res2 = await contact.findById(res1.id)
expect(res2).toMatchObject(res1)
})
it('delete all, create a few, select all', async () => {
const faker = require("faker")
const contact = require("../../models/contact-us.model")
await contact.removeAll()
const total = 5
const contacts = []
for (let i = 0; i < total; i++) {
const req = {
first_name: faker.name.firstName(),
email: faker.internet.email(),
message: faker.lorem.sentences(5),
last_name: faker.name.lastName(),
phone: faker.phone.phoneNumberFormat(),
address: faker.address.streetAddress("###")
}
const res1 = await contact.create(req)
contacts.push(res1)
}
const all = await contact.getAll()
expect(all.length).toBe(total)
for (let i = 0; i < total; i++) {
expect(all[i]).toMatchObject(contacts[i])
}
})
afterAll(() => {
const db = require("../../models/db")
db.destroy();
})
})
Here, first, I insert a new object and query it immediately. Than I remove all records, insert a few new, generating them using faker, query them all again and close the database connection.
After models are tested, one may test the rest of application without MySql container, just mocking MySql and models using Jest:
jest.mock("mysql", () => {
return {
createConnection: () => ({
connect: () => null
})
}
})
const model = require("../../models/contact-us.model")
jest.mock("../../models/contact-us.model")
const mockResponse = () => {
const res = {}
res.send = jest.fn().mockReturnValue(res)
res.status = jest.fn().mockReturnValue(res)
res.json = jest.fn().mockReturnValue(res)
return res
}
describe('controllers', () => {
it('contact creation success', async () => {
const data = {
first_name: 'name',
email: 'email@email.com',
message: 'message',
last_name: 'surname',
phone: 'phone',
address: 'address'
}
const contact = require("../../controllers/contact-us.controller")
const res = mockResponse ()
await contact.create({ body: data }, res)
expect(model.create.mock.calls.length).toBe(1)
expect(model.create.mock.calls[0][0]).toEqual(data)
expect(res.send.mock.calls[0][0]).toEqual({message: "Success!"})
})
})
Next
As one may easily see, model code is not especially original, and could be easily distilled into more generic functions for easy reuse with similar tables. As our project and the number of endpoints grow, we often find ourself in a situation, when our application constantly reuses just a small set of core, generic functions and UI components. Adding new endpoints becomes practically the matter of configuration. In such situations, often, tests become less and less helpful, as they essentially retest our generic functions with different parameters. At the same time, as our production infrastructure becomes more and more and more complex, interactions between different components of application become the main source of bugs and complexity. At this point we may wish to resort to the integration and end-to-end testing and to the tool simplifying such tests (like Cypress). Even than, clean project structure and ability to test different application components in isolation when necessary (an approach advocated in Double Loop Testing) greatly simplify maintenance and further development.