- Software experts
NGINX Tutorial: Developing Modules
Table of contents.
- 1 Introduction
- 2.1 Getting started with the NGINX tutorial
- 2.2 Bootstrapping a local development environment
- 2.3 Ensure the local environment is working properly
- 3.1 Creating the build system
- 3.2 Create a simple wiring test with RSpec for the environment
- 4 Hello NGINX
- 4.2 Giving it a spin
- 5 Finding the authentication token in the request
- 6.1 Querying Redis
- 7 Handling the scenarios
- 8 Adding configuration directives
- 9 Updating the test suite
- 10 Adding a backing application
- 11 Closing thoughts
##1 Introduction
This NGINX tutorial and the accompanying video will be a look into developing modules for the NGINX web server. Typically we use web servers like NGINX and Apache as simple reverse proxies for our web based software, leaving a lot of functionality on the table. We will explore not only how to build an NGINX module, but also build something that could provide value to a web application by allowing NGINX to handle some of the early routing process.
##2 Setting up the environment
NGINX modules can be written in a number of languages. The most common are C, Perl, and Lua. This tutorial will use C as the implementation language and Ruby as a supporting language. It offers the greatest amount of control and integration with NGINX and can make a big difference in the performance department. Writing NGINX modules in C also helps you internalize the NGINX life-cycle and how it operates on requests.
###2.1 Getting started with the NGINX tutorial
Before we start there are a few things we will need in order to compile our module. Since we typically deploy to Linux this tutorial will work directly in a Linux environment. If you don't have one setup, everything here should work on OS X but the dependency install process will be a little bit different.
First, you will need a C compiler and other C based build tools. The following packages will satisfy all dependencies for this NGINX tutorial:
Along with these dependecies you will need Ruby, Rubygems, and Bundler. Ruby installers such as RVM or rbenv provide everything in one place.
###2.2 Bootstrapping a local development environment
In order to work effectively, we first need to setup a proper development environment. Since NGINX doesn't have a shared module system, it will have to be recompiled every time we want to try the module out. This can be cumbersome if we install NGINX to it's normal location. This requires root privileges when editing configuration files and starting/stopping the server. You might also have another web server running on your machine, which could create a conflict for standard tcp ports. We also need a minimal configuration so we can focus on our module. The following nginx.conf is the bare minimum configuration:
script/bootstrap
This configuration will run NGINX on port 8888 . Now that we have a configuration, we need to download and install nginx. Since we can control where nginx gets installed, we will set everything up in the same folder that we write our module code. Rather than go into a lot of detail about how to do this, the following script can be used to download and install NGINX to the proper location:
Along with installing NGINX, this script will create a symlink on the nginx.conf file that was just created. Make sure to make it executable and give it a whirl.
###2.3 Ensure the local environment is working properly
Now we are ready to test everything out and make sure it is working properly. Try the following to ensure everything worked. If you see the following then you are successfully setup and ready to go!
##3 Establishing a test harness
As we develop our module we will want to make sure that the results are as expected. Let's setup a build and test harness to help keep us in line as we write our code.
###3.1 Creating the build system
There are a lot of potential options for controlling the build and test environment. At the end of the day the best option is the one that works for you. For this exercise we will be using the Ruby language and the RSpec test framework. We will use bundler to deal with dependencies and Rake to drive compilation and test runs. We will use the curb gem to perform the calls to our web server and the redis gem to communicate with Redis. It all starts with a Gemfile:
To fetch these dependencies we need to run bundler:
Now that we have our dependencies installed let's move on to our Rakefile .
This will allow us to drive everything via Rake tasks. To ensure everything was wired up correctly you can run rake -T . You should see the following output:
You might have noticed that we are missing the file referenced in the compile task. Let's sort that out now:
script/compile
This is similar to the bootstrap compilation routine, but it adds debug flags and disables optimizations to make troubleshooting easier. Don't forget to make it executable. Let's give the compile task a try. We will use this going forward so it's a good idea to make sure it is working properly.
###3.2 Create a simple wiring test with RSpec for the environment
With dependencies and a Rakefile in place, the last thing to do before we get started on our module is write the base for our test harness. The first test we write will just ensure that the web server is running and returns an HTTP 200 OK response. First we will create our spec helper file which will be included in all of our tests:
_spec/spec helper.rb
There isn't a lot here and that's ok. It just requires the dependencies for our spec files. As you progress down the road this can offer a lot more. Next we will create our actual spec file:
_spec/integration/integration spec.rb
You can run your test using the rake command:
This shows that our test passed, but doesn't quite give us the information we want. We can change this by adding a .rspec file to our project:
Running the test now gives us a more human readable output:
Now it's time to write our module.
##4 Hello NGINX
NGINX modules consist of two parts; a config file and a module file. The config file tells NGINX how to compile the module and what it is called. The module file contains the actual code. The module file can of course be broken up into multiple files, but we will only have one for this exercise.
Let's start with the config file:
This file tells the NGINX compilation process what the module name is and where the source code can be found. We will come back to this file later to add some additional items but for now this is all we need.
##4.1 Module code
Now it's time for our actual module code.
_ngx_http_auth_token module.c
There are a few things we need to include to write our module. The four included files come from NGINX and will eventually all be necessary to complete our module. ngx_http_auth_token_module is declared as our module.
ngx_http_auth_token_handler is the function that runs when the request is processed. We will see below how it is wired into the request life-cycle, but it is the function responsible for coordinating all of the actions in this module. The function accepts the parsed request and operates on it.
First, we check to see if this function has already been invoked. NGINX has multiple phases and can sometimes invoke a handler more than once. If we see that the internal flag has been set we stop the handler from continuing by returning NGX_DECLINED . This seems counter intuitive, but that response tells NGINX to keep going. If the internal flag has not been set, the next operation sets it to mark that processing has already occurred.
The meat of this function creates a new entry for the response headers. Once the new header has been added, the function returns NGX_DECLINED to let NGINX know to continue to the next phase of processing.
The init function is responsible for wiring the handler function tovthe proper phase in the NGINX life-cycle. It gets the core nginxvconfiguration struct and creates an entry in the NGX_HTTP_ACCESS_PHASE . Finally it sets the function in the handler to the ngx_http_auth_token_handler function.
Last, we need to configure the module itself. The final two values specify how the module should be invoked. Right now most of these options are not used because our module is pretty light. Once we add configuration directives and additional code we will add on these. Running through these options we see the ngx_http_auth_token_init function referenced for the postconfiguration section. This tells NGINX to call this function when loading the module. The module declaration itself comes next. Again, we don't do much here. We use the boilerplate values and pass our ngx_http_auth_token_module_ctx as the module context argument.
###4.2 Giving it a spin
You will notice if you run rake nginx:compile that it does not compile our new module. This is because we have not included it in our compile script. Let's update that now:
This tells the NGINX compilation process to look for a module in the ../../ directory. This is accurate since the NGINX compilation process is running in the vendor/nginx-1.4.7 directory. After the compilation succeeds, give it a try:
Notice the new header X-NGINX-Tutorial: Hello World! is now in the response. We don't actually want to keep this in the response going forward, but this is an easy way to demonstrate that our code is included and running properly.
##5 Finding the authentication token in the request
Now that we have a working module it's time to write the real part of our module. The first part of which is finding the value of a cookie. In particular, we want to find the value of the cookie that contains an authentication token. To do this we will leverage a built in NGINX function called ngx_http_parse_multi_header_lines . The code is as follows:
This function takes the cookies from the request, the name of the cookie to extract, and a variable to store the value of the cookie. Right now the name of the cookie is hard coded. We will fix that when we add configuration directives. Let's update our handler to extract the value and redirect if the cookie is not found.
After adding the cookie extraction code, we added an if statement to either display the header back to the user or redirect if the header isn't present. We will continue by adding some lookup code to validate that the auth token is in fact legitimate. Let's recompile and give it a try.
##6 Looking up the authentication token in Redis
The next step is to lookup the auth token provided and ensure that it is valid. For this exercise, valid simply means that the token can be found. We will use Redis as our token storage system. There are plenty of other options, and some that work better in this scenario. Redis has a clean and easy to use C library, and makes for a lot less code. It is perfectly acceptable to use Redis in this situation.
###6.1 Querying Redis
The first thing we need is a connection to Redis. This can be achieved via the redisConnect or redisConnectWithTimeout functions. Normally, we would want to use the timeout feature to ensure that the request can fail quickly if there is a problem downstream with Redis or the network. For this example we will use redisConnect .
This populates a context struct with the Redis connection. We will pass this along when we make queries. Let's write the function we will use in our module to retrieve the user id using the auth token value. Before we write our funciton we need to include a couple more files at the top of our module.
Now we can write our lookup function:
This function accepts the auth token value from the request and a variable to hold the user id if it is found. A connection is made to redis and passed in to the redisCommand function along with the query GET <auth_token> . If the query succeeds, a string will be returned. If it fails, the reply object will have a reply type of REDIS_REPLY_NIL . We can use that to signal that the lookup failed. If so, we return NGX_DECLINED . This lets the caller know that the value didn't exist. If the lookup succeeded we need to populate the user_id variable. In NGINX, strings are a little different. A ngx_str_t consists of data and a length. They aren't null terminated. When writing NGINX modules it is best to work with their data structures. To make things easier, all functions in your module should speak NGINX types and do any necessary conversions inside. We set the data member of user_id to the reply value from Redis and cast it to a u_char * . We then set the len member to the length of the reply string and return NGX_OK .
We now have a function capable of performing our lookup. There is one more thing we need to do to have our module compile properly. We need to update our config and tell it that we want to link against the hiredis library. Here's what it should look like when we are finished:
Here we updated CORE_LIBS to include -lhireds . This will ensure that the compilation process will link against the hiredis library and our compile will succeed. Even though we haven't hooked up our new function, it's a good idea to test the compilation process now to make sure everything works properly. If you get errors, make sure the hiredis library is installed and that pkg-config is aware of it.
##7 Handling the scenarios
We are now able to retrieve the value of a cookie and look it up in Redis. The last thing our module needs is some logic. We want to redirect if the request doesn't contain the auth token, or if the auth token isn't found. If it is found, we want to allow the request to continue. In order to do this effectively, we are going to want to do a little refactoring. Let's start by extracting the two paths of our handler into functions.
First, we create a redirect function. We will be using it twice, so it's nice to have it tucked away for reuse. This is just a function extraction, nothing changed from the original code. Next, let's extract the code that's left:
Here's where things change a bit. This function takes the user id that we looked up from Redis and appends it as the value in our new header. Notice that the name of the header has changed from X-Auth-Token to X-User-Id . This is because the value we will eventually pass to our downstream application is the user id. Now we can rewrite our handler to use these new functions and put in place the logic we want.
The consequence of the cookie lookup now just invokes our redirect function. The alternative attempts to lookup the auth token using the lookup_user function. If it fails the redirect function is invoked. If it passes we invoke our append_user_id function and exit our handler. Let's recompile and test out our module.
We can see that we are being redirected in both situations where we expect. Let's add the test key into Redis and see what happens.
This time the lookup was successful and the X-User-Id header was set to the value of the test key.
This is the last time we are going to see the new header reflected back to us. It isn't useful or necessary to add it to the response. We need to make sure that the application that comes after our NGINX server sees this header. Let's modify our append_user_id function to make this happen.
If we recompile and test again, we will see that the header is no longer in the response. To test this now we will need to add a backing application. We won't do that quite yet though, there's still some finishing touches we need to put on our module.
##8 Adding configuration directives
Until now all of the values in our module were hardcoded. This doesn't make a great module. There are a few things we would like to configure:
- The Redis host
- The Redis port
- The cookie name for the auth token
- The location of the redirect
To get the idea of how we want the end result to look, let's update our nginx.conf file to reflect the desired state.
This example shows what our configuration will look like when we are finished. In NGINX there are multiple places that you can have configuration variables. The section in the http block is referred to as the NGX_HTTP_MAIN_CONF , the server block NGX_HTTP_SRV_CONF , and the location block NGX_HTTP_LOC_CONF . There are additional options that won't be discussed in this NGINX tutorial. For a full list you can visit http://lxr.nginx.org/source/src/http/ngx_http_config.h . Directives are allowed to be declared in any combination of locations. To make things a little clearer we will only use NGX_HTTP_MAIN_CONF in this example. One might argue that it would make more sense to have them in other places. While that is correct this is an exercise left to the reader.
To get started we need to create a place to hold the configuration. We will declare a struct at the top of our module.
Next, we need to add the definition for our directives.
This section of code is pretty straight forward. For each directive you have a name, the configuration location, the number of arguments it accepts, the type of argument, the configuration offset, and the value in the configuration struct to populate. There are just two more things left to do and we are ready to test it out.
Next, we need to tell the NGINX how to find these commands and what to default them to. First lets wire them in:
Note that the module directives has changed from NULL to the command list we just created. Along with this we added a reference to a function named ngx_http_auth_token_create_main_conf . This function will allocate and set default values for our configuration struct. Let's write that now.
This code allocates the auth_token_main_conf_t value to ensure that it is usable by the module. Now we need to update our code to use the configuration values. Let's update our handler.
We have now removed the hardcoded values. In their place we created an instance of our auth_token_main_conf_t and populated it using the ngx_http_get_module_main_conf function. After this we are able to use the conf variable to get to our configuration values. There is one more change that we still have to make. We added an extra argument to the lookup_user function. This is the configuration information so that the function can connect to Redis using the configuration values rather than the hardcoded ones. Let's update the function to reflect the new signature and remove the hardcoded data.
These changes are pretty straight forward. We add conf to the function signature and use its values inside the function. Remember that the conf contains NGINX data structures, so to get string values you need to use the data member of the value. Now we're ready to give it a try!
##9 Updating the test suite
Instead of firing everything up and running a bunch of curl commands, let's update our test suite to do this automatically.
_spec/integration/spec helper.rb
Before each test we reset our Redis database. This will make sure we only have the values we want inside of each of the tests. We then create three tests that express the boundaries of our module. We check the HTTP status code of the response to ensure that the module is acting the way we intend it to. Let's recompile and run our tests.
Now that we have a working test suite we have a nice base for future additions to our module.
##10 Adding a backing application
In order to see the results of our module in action we need to have an application running behind our NGINX server. It doesn't need to be fancy. In fact, it really doesn't need to do much of anything except print the user id so that we can see everything working as expected. We already have an environment setup for Ruby, so let's create a simple Ruby web application that does just this.
This program will respond if the proper auth token is presented to NGINX. Before we can test everything we need to update our NGINX configuration to add our new program as the receiver of our requests.
We added an upstream block that points to our local host at port 4567, which is where our program will run by default. Now we just need to start NGINX and our new program:
Let's make sure we have a token in our Redis database then query the server.
We can see that our module is correctly passing the information back to the downstream application.
##11 Closing thoughts
We now have a fully functioning NGINX module complete with test suite. We accomplished a lot and hopefully learned some new things. There are lots of places we can go with this module but there are a few things we should discuss before wrapping up.
- Should I use this module in production? Probably not. It is lacking some flexibility in configuration and general robustness and error handling. Along with that this module is still pretty plain. Additional features may be necessary to not take away from the intended security mechanisms that may already be in place for your application. That being said, this is a solid foundation and the rest of the work doesn't take too much additional effort.
- Where do I go from here? There are quite a few resources available. The general go to place for information is the mailing lists . There is also the canonical guide to NGINX module writing from Evan Miller. It's a great resource for general information about getting started writing modules. Finally, you should experiment with this module and see what else you can add to it and share what you have created with the world!
Similar posts All posts
python nginx apache All about servers
All you need to know about servers as a python developer.
python docker nginx devops In this post I will describe how to use Docker to plug together Consul, Consul Template, Registrator and Nginx into a truly scalable architecture that I am calling DR CoN. Once all plugged together, DR CoN lets you add and remove services from the architecture without having to rewrite any configuration or restart any services, and everything just works!
Scalable Architecture DR CoN: Docker, Registrator, Consul, Consul Template and Nginx
nginx apache Web performance expert Regis Leroy discusses strategies for adapting Apache behaviors to Nginx.
The Ultimate Guide to Migrating From Apache to Nginx: Part 2
- Stack Overflow Public questions & answers
- Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers
- Talent Build your employer brand
- Advertising Reach developers & technologists worldwide
- About the company
Collectives™ on Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most.
Q&A for work
Connect and share knowledge within a single location that is structured and easy to search.
How to write a Nginx module? [closed]
I'm trying to find tutorials on how to build a module/plugin for Nginx web server.
Can someone help please, I just can't seem to find the appropriate tutorials.
- nginx-module
3 Answers 3
Quoting from the documentation:
Evan Miller has written the definitive guide to Nginx module development . But some parts of it are a little out of date. You've been warned. A github search turned up the Nginx Development Kit . It seems to be more up to date.
From my own personal experience, Evan Miller's guide was of a great help. You must also have a deep understanding of how NGINX works. Agentzh's tutorial can help you.
Reading the source code of his modules is always helpful too.
There is also a video tutorial that I haven't check yet, but it seems nice.
http://www.evanmiller.org/nginx-modules-guide.html
Nginx has a module chain . When Nginx needs to gzip or chunk-encode a response, it whips out a module to do the work. When Nginx blocks access to a resource based on IP address or HTTP auth credentials, a module does the deflecting. When Nginx communicates with Memcache or FastCGI servers, a module is the walkie-talkie... The purpose of this guide is to teach you the details of Nginx's module chain... When you're done with the guide, you'll be able to design and produce high-quality modules that enable Nginx to do things it couldn't do before. Nginx's module system has a lot of nuance and nitty-gritty, so you'll probably want to refer back to this document often. I have tried to make the concepts as clear as possible, but I'll be blunt, writing Nginx modules can still be hard work...
- have you made any progress on that? I'm kind of in the same situation here. Take a look at my post: stackoverflow.com/questions/4204812/… – PartySoft Nov 17, 2010 at 13:35
- The bit I felt wasn't as explicit as it should be in an otherwise fantastic post was this: 'Finally, add a module directive to nginx's configuration (default is /usr/local/nginx/conf/nginx.conf) to enable the module for a location. location = /hello { hello_world; }' – jbg Jul 5, 2011 at 7:42
You can find an official nginx development guide here .
The end of the document leads to a new repo (6 weeks before this post) full of examples here .
Not the answer you're looking for? Browse other questions tagged nginx plugins nginx-module or ask your own question .
- The Overflow Blog
- Building an API is half the battle: Q&A with Marco Palladino from Kong
- Developers think AI assistants will be everywhere, but aren’t sure how to...
- Featured on Meta
- We've added a "Necessary cookies only" option to the cookie consent popup
- The Stack Exchange reputation system: What's working? What's not?
- Launching the CI/CD and R Collectives and community editing features for...
- The [amazon] tag is being burninated
- Staging Ground Beta 1 Recap, and Reviewers needed for Beta 2
- Temporary policy: ChatGPT is banned
Hot Network Questions
- How were rackmount workstations wired-up to mice, keyboards, monitors, etc?
- Looking for a possible trilogy from the 2000s
- Is it legal for a company to require you to delete your account to unsubscribe from marketing emails?
- Tools/techniques to remove a long section from a wooden board
- Including somebody with an unrelated degree as a coauthor?
- Portable Alternatives to Traditional Keyboard/Mouse Input
- How do I get the shader node type in python?
- Is it OK practice to start a car while it's on jackstands?
- Did mechanical hard drives often malfunction in high elevation places such as Bogota?
- DFA that accepts strings whose 10th symbol from the right end is 1
- Was Freemasonry such a big problem in 1980s UK policing?
- Why sulfites are often ignored in soil studies?
- Best way to highlight the main result in a mathematical paper
- "Da" position in subordinate clauses
- broker cancelled life insurance policy without authorization
- Using vegetable peelings to make stock
- Training for driving on the left in the UK
- Movie where a man is abducted by beings believed to be aliens and finds out they were from the future
- Why isn't the derivative of the volume of the cone its surface area?
- Extracting list elements following a specific marker
- Meaning of も in ３つ も
- Is there really a bawdy pun at the conclusion of Romeo and Juliet?
- "Hierba" or "Yerba" - which is gramatically correct?
- Colin Farrell Got The Last Laugh On “Saturday Night Live” By Shading Their “Offensive” Sketch About Irish People At The Oscars
Your privacy
By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy .
Analytics cookies are off for visitors from the UK or EEA unless they click Accept or submit a form on nginx.com. They’re on by default for everybody else. Follow the instructions here to deactivate analytics cookies. This deactivation will work even if you later click Accept or submit a form.
Check this box so we and our advertising and social media partners can use cookies on nginx.com to better tailor ads to your interests. Uncheck it to withdraw consent.
Cookie preferences
Accept cookies for analytics, social media, and advertising, or learn more and adjust your preferences . These cookies are on by default for visitors outside the UK and EEA. Privacy Notice .
- Partner Central
- F5 NGINX Sales
- F5 Professional Services
Resources > Blog > Tech
Compiling Third-Party Dynamic Modules for NGINX and NGINX Plus
Editor – The blog post announcing the original release of dynamic module support (in NGINX Open Source 1.9.11, February 2016) redirects here. The build process described in that post is deprecated.
This post is part of a two‑part series on using third‑party dynamic modules with NGINX Open Source and NGINX Plus.
- This post provides step-by-step instructions for compiling third‑party dynamic modules which can be loaded by NGINX Open Source or NGINX Plus at runtime.
- The second post provides guidance and tooling for automating third‑party dynamic module builds for a production environment. It explains how to create installable packages for third‑party dynamic modules that include version dependency checking.
NGINX Open Source 1.11.5 and NGINX Plus Release R11 introduced binary compatibility for dynamic modules. This article explains how to compile third‑party modules for use with NGINX Open Source and NGINX Plus in a development environment. For instructions on building, deploying, and upgrading third‑party dynamic modules in a production environment, see Creating Installable Packages for Dynamic Modules .
For the sake of brevity, the remainder of this post refers to NGINX Plus only, except when the difference between it and NGINX Open Source is relevant. Except as noted, all statements about NGINX Plus also apply to NGINX Open Source.
Dynamic Modules Overview
Modules that can be loaded into NGINX Plus are written in C and conform to the API described in Extending NGINX on the NGINX Wiki. There is a large ecosystem of third‑party modules , ranging from language interpreters to security solutions, and some of these are included and supported in NGINX Plus.
Other third‑party modules, and modules that you have created yourself, need to be compiled independently and dynamically loaded into NGINX Plus at runtime. You can compile these modules for use with NGINX Plus by building them against NGINX Open Source as illustrated in the two examples below:
- Obtain the matching NGINX Open Source release
- Obtain the module sources and change the module’s config file if necessary
- Build the dynamic module against the NGINX Open Source release, with the --with-compat argument to the configure command
- Load the resulting dynamic module (the .so file) into NGINX Plus and use it as if it were a built‑in module
Example: A Simple “Hello World” Module
This example uses a simple Hello World module to show how to update the source for a module and load it into NGINX Plus. The “Hello World” module implements a simple directive ( hello_world ) that responds to requests with a simple message.
Step 1: Obtain the NGINX Open Source Release
Determine the NGINX Open Source version that corresponds to your NGINX Plus installation. In this example, it’s NGINX 1.11.5.
Download the corresponding NGINX Open Source package at nginx.org/download :
Step 2: Obtain the Module Sources
Obtain the source for the ‘Hello World’ NGINX module from GitHub :
The config shell file for a module defines how it is built, and its format is different for dynamic modules than for modules being built statically into an NGINX Open Source binary.
Modify the file nginx-hello-world-module/config to contain the following:
For detailed information about compiling dynamic modules, including instructions on updating a module’s config file from the old format, see the NGINX Wiki .
Step 3: Compile the Dynamic Module
Compile the module by first running the configure script with the --with-compat argument, which creates a standard build environment supported by both NGINX Open Source and NGINX Plus. Then run make modules to build the module:
Copy the module library ( .so file) to /etc/nginx/modules :
Step 4: Load and Use the Module
To load the module into NGINX Plus, add the load_module directive in the top‑level (main) context of your nginx.conf configuration file (not within the http or stream context):
In the http context, add a location block with the hello_world directive provided by the Hello World module. Requests to the location return the response hello world .
Reload your NGINX Plus configuration and test it with a simple request:
Example: The NAXSI Web Application Firewall
NAXSI is an easy‑to‑use, high‑performance web application firewall (WAF) that uses heuristics and a scoring system to identify suspicious requests such as XSS and SQL Injection attacks.
The NAXSI source has been updated to comply with the new format for the config shell file, so building a dynamic module for NGINX Plus is straightforward. The process is based on the NAXSI installation instructions :
Load the module into the NGINX Plus core by adding the load_module directive to the main context in your nginx.conf file:
NAXSI configuration is described in detail in the project documentation . The following NGINX configuration illustrates the module in action:
You can verify the correct operation of NAXSI with a pair of simple HTTP requests:
- curl http://localhost/ returns the standard NGINX Plus index page stored in /usr/share/nginx/html .
- curl "http://localhost/?a=<>" triggers NAXSI’s XSS detection and blocks the request, returning the standard 50x.html error page from /usr/share/nginx/html . It also logs a message to the error_log .
For production deployments, you can also download signed NAXSI releases at https://github.com/nbs-system/naxsi/tags and compile them in a similar fashion.
How We Support Dynamic Modules in NGINX Plus
Note: The information in this section applies to NGINX Plus only. The set of dynamic modules that ships with prebuilt NGINX Open Source packages might differ from the ones shipped with NGINX Plus. Dynamic modules used with NGINX Open Source are supported in the same way as the NGINX source code and prebuilt binaries.
NGINX Plus ships with a number of dynamic modules that you can also download directly from our modules repository. For a list, see the Dynamic Modules page. These modules are of two types:
- NGINX Plus modules are written and/or maintained by the NGINX engineering team. We don’t include them in NGINX Plus for technical reasons (for example, they have additional dependencies) or because they are in a preview state. Preview modules are in active development and should only be deployed with great care. Otherwise, NGINX Plus modules are fully supported by NGINX. For a list, filter by author NGINX on the Dynamic Modules page.
- NGINX Plus certified community modules are popular third‑party modules that NGINX tests and distributes, and for which we provide support with installation and basic configuration. We warrant that these modules do not interfere with the correct operation of NGINX Plus, and we update them as necessary at each NGINX Plus release or when there is a security release. For a list, filter by author Community on the Dynamic Modules page.
In addition, NGINX certifies modules from commercial vendors who participate in our NGINX Plus Certified Modules program. These modules are distributed and supported by their vendors. For a list, filter by author Certified Partner on the Dynamic Modules page.
NGINX does not test or support modules that you compile yourself (other community modules, modules provided by third‑party vendors that are not part of the NGINX Plus Certified Modules program, and custom modules). If you seek technical support for a problem, the NGINX support team may ask you to remove an unsupported module and reproduce the fault as part of our technical support process, so that they can verify whether or not the fault is caused by the unsupported module.
The dynamic modules build process for NGINX Plus allows you to take advantage of the broad ecosystem of NGINX Open Source modules, running them on the rich and fully supported NGINX Plus core.
If you are currently using NGINX Open Source with third‑party extensions, these extensions can most likely be compiled and loaded into NGINX Plus.
If you develop commercial or community modules, the new build process means your users can deploy your modules with NGINX Plus. To learn about certifying a commercial module, see NGINX Plus Certified Modules .
If you need assistance in developing a module or updating its config shell file, please check out the following resources:
- Emiller’s Guide to NGINX Module Development by Evan Miller
- Dynamic Modules Development by NGINX developer Ruslan Ermilov
- Extending NGINX on the NGINX Wiki
- Converting Static Modules to Dynamic Modules on the NGINX Wiki
- NGINX Tutorial: Developing Modules by Aaron Bedra
- Creating Installable Packages for Dynamic Modules on our blog
The NGINX Developers mailing list is the go‑to place for community assistance, and our Professional Services team is also happy to assist.
To try dynamic modules with NGINX Plus yourself, start a free 30-day trial today or contact us to discuss your use cases .
Updated for 2022 – Your Guide to Everything NGINX
About The Author
Owen Garrett
Sr. Director, Product Management
Owen is a senior member of the NGINX Product Management team, covering open source and commercial NGINX products. He holds a particular responsibility for microservices and Kubernetes‑centric solutions. He’s constantly amazed by the ingenuity of NGINX users and still learns of new ways to use NGINX with every discussion.
More Blogs By Owen Garrett
About F5 NGINX
F5, Inc. is the company behind NGINX, the popular open source project. We offer a suite of technologies for developing and delivering modern applications. Together with F5, our combined solution bridges the gap between NetOps and DevOps, with multi-cloud application services that span from code to customer.
Learn more at nginx.com or join the conversation by following @nginx on Twitter.
Get Started
Try out nginx plus on your turf, ask us a question.
Emiller’s Guide To Nginx Module Development
By Evan Miller
First published: April 28, 2007 (Last edit: August 11, 2017 – changes )
To fully appreciate Nginx, the web server, it helps to understand Batman, the comic book character.
Batman is fast. Nginx is fast. Batman fights crime. Nginx fights wasted CPU cycles and memory leaks. Batman performs well under pressure. Nginx, for its part, excels under heavy server loads.
But Batman would be almost nothing without the Batman utility belt .
Figure 1 : The Batman utility belt, gripping Christian Bale’s love handles.
At any given time, Batman’s utility belt might contain a lock pick, several batarangs, bat-cuffs, a bat-tracer, bat-darts, night vision goggles, thermite grenades, smoke pellets, a flashlight, a kryptonite ring, an acetylene torch, or an Apple iPhone. When Batman needs to tranquilize, blind, deafen, stun, track, stop, smoke out, or text-message the enemy, you better believe he’s reaching down for his bat-belt. The belt is so crucial to Batman’s operations that if Batman had to choose between wearing pants and wearing the utility belt, he would definitely choose the belt. In fact, he did choose the utility belt, and that’s why Batman wears rubber tights instead of pants (Fig. 1).
Instead of a utility belt, Nginx has a module chain . When Nginx needs to gzip or chunk-encode a response, it whips out a module to do the work. When Nginx blocks access to a resource based on IP address or HTTP auth credentials, a module does the deflecting. When Nginx communicates with Memcache or FastCGI servers, a module is the walkie-talkie.
Batman’s utility belt holds a lot of doo-hickeys, but occasionally Batman needs a new tool. Maybe there’s a new enemy against whom bat-cuffs and batarangs are ineffectual. Or Batman needs a new ability, like being able to breathe underwater. That’s when Batman rings up Lucius Fox to engineer the appropriate bat-gadget.
Figure 2 : Bruce Wayne (née Batman) consults with his engineer, Lucius Fox
The purpose of this guide is to teach you the details of Nginx’s module chain, so that you may be like Lucius Fox. When you’re done with the guide, you’ll be able to design and produce high-quality modules that enable Nginx to do things it couldn’t do before. Nginx’s module system has a lot of nuance and nitty-gritty, so you’ll probably want to refer back to this document often. I have tried to make the concepts as clear as possible, but I’ll be blunt, writing Nginx modules can still be hard work.
But whoever said making bat-tools would be easy?
Table of Contents
- Prerequisites
- High-Level Overview of Nginx’s Module Delegation
- Components of an Nginx Module
- Module Configuration Struct(s)
- Module Directives
- The Module Context
- create_loc_conf
- merge_loc_conf
- The Module Definition
- Module Installation
- Anatomy of a Handler (Non-proxying)
- Getting the location configuration
- Generating a response
- Sending the header
- Sending the body
- Anatomy of an Upstream (a.k.a. Proxy) Handler
- Summary of upstream callbacks
- The create_request callback
- The process_header callback
- Keeping state
- Handler Installation
- Anatomy of a Header Filter
- Anatomy of a Body Filter
- Filter Installation
- Load-Balancers
- The enabling directive
- The registration function
- The upstream initialization function
- The peer initialization function
- The load-balancing function
- The peer release function
- Writing and Compiling a New Nginx Module
- Advanced Topics
- Code References
0. Prerequisites
You should be comfortable with C. Not just "C-syntax"; you should know your way around a struct and not be scared off by pointers and function references, and be cognizant of the preprocessor. If you need to brush up, nothing beats K&R .
Basic understanding of HTTP is useful. You’ll be working on a web server, after all.
You should also be familiar with Nginx’s configuration file. If you’re not, here’s the gist of it: there are four contexts (called main , server , upstream , and location ) which can contain directives with one or more arguments. Directives in the main context apply to everything; directives in the server context apply to a particular host/port; directives in the upstream context refer to a set of backend servers; and directives in a location context apply only to matching web locations (e.g., "/", "/images", etc.) A location context inherits from the surrounding server context, and a server context inherits from the main context. The upstream context neither inherits nor imparts its properties; it has its own special directives that don’t really apply elsewhere. I’ll refer to these four contexts quite a bit, so… don’t forget them.
Let’s get started.
1. High-Level Overview of Nginx’s Module Delegation
Nginx modules have three roles we’ll cover:
- handlers process a request and produce output
- filters manipulate the output produced by a handler
- load-balancers choose a backend server to send a request to, when more than one backend server is eligible
Modules do all of the "real work" that you might associate with a web server: whenever Nginx serves a file or proxies a request to another server, there’s a handler module doing the work; when Nginx gzips the output or executes a server-side include, it’s using filter modules. The "core" of Nginx simply takes care of all the network and application protocols and sets up the sequence of modules that are eligible to process a request. The de-centralized architecture makes it possible for you to make a nice self-contained unit that does something you want.
Note: Unlike modules in Apache, Nginx modules are not dynamically linked. (In other words, they’re compiled right into the Nginx binary.)
How does a module get invoked? Typically, at server startup, each handler gets a chance to attach itself to particular locations defined in the configuration; if more than one handler attaches to a particular location, only one will "win" (but a good config writer won’t let a conflict happen). Handlers can return in three ways: all is good, there was an error, or it can decline to process the request and defer to the default handler (typically something that serves static files).
If the handler happens to be a reverse proxy to some set of backend servers, there is room for another type of module: the load-balancer. A load-balancer takes a request and a set of backend servers and decides which server will get the request. Nginx ships with two load-balancing modules: round-robin, which deals out requests like cards at the start of a poker game, and the "IP hash" method, which ensures that a particular client will hit the same backend server across multiple requests.
If the handler does not produce an error, the filters are called. Multiple filters can hook into each location, so that (for example) a response can be compressed and then chunked. The order of their execution is determined at compile-time. Filters have the classic "CHAIN OF RESPONSIBILITY" design pattern: one filter is called, does its work, and then calls the next filter, until the final filter is called, and Nginx finishes up the response.
The really cool part about the filter chain is that each filter doesn’t wait for the previous filter to finish; it can process the previous filter’s output as it’s being produced, sort of like the Unix pipeline. Filters operate on buffers , which are usually the size of a page (4K), although you can change this in your nginx.conf. This means, for example, a module can start compressing the response from a backend server and stream it to the client before the module has received the entire response from the backend. Nice!
So to wrap up the conceptual overview, the typical processing cycle goes:
Client sends HTTP request → Nginx chooses the appropriate handler based on the location config → (if applicable) load-balancer picks a backend server → Handler does its thing and passes each output buffer to the first filter → First filter passes the output to the second filter → second to third → third to fourth → etc. → Final response sent to client
I say "typically" because Nginx’s module invocation is extremely customizable. It places a big burden on module writers to define exactly how and when the module should run (I happen to think too big a burden). Invocation is actually performed through a series of callbacks, and there are a lot of them. Namely, you can provide a function to be executed:
- Just before the server reads the config file
- For every configuration directive for the location and server for which it appears;
- When Nginx initializes the main configuration
- When Nginx initializes the server (i.e., host/port) configuration
- When Nginx merges the server configuration with the main configuration
- When Nginx initializes the location configuration
- When Nginx merges the location configuration with its parent server configuration
- When Nginx’s master process starts
- When a new worker process starts
- When a worker process exits
- When the master exits
- Handling a request
- Filtering response headers
- Filtering the response body
- Picking a backend server
- Initiating a request to a backend server
- Re -initiating a request to a backend server
- Processing the response from a backend server
- Finishing an interaction with a backend server
Holy mackerel! It’s a bit overwhelming. You’ve got a lot of power at your disposal, but you can still do something useful using only a couple of these hooks and a couple of corresponding functions. Time to dive into some modules.
2. Components of an Nginx Module
As I said, you have a lot of flexibility when it comes to making an Nginx module. This section will describe the parts that are almost always present. It’s intended as a guide for understanding a module, and a reference for when you think you’re ready to start writing a module.
2.1. Module Configuration Struct(s)
Modules can define up to three configuration structs, one for the main, server, and location contexts. Most modules just need a location configuration. The naming convention for these is ngx_http_<module name>_(main|srv|loc)_conf_t . Here’s an example, taken from the dav module:
Notice that Nginx has special data types ( ngx_uint_t and ngx_flag_t ); these are just aliases for the primitive data types you know and love (cf. core/ngx_config.h if you’re curious).
The elements in the configuration structs are populated by module directives.
2.2. Module Directives
A module’s directives appear in a static array of ngx_command_t s. Here’s an example of how they’re declared, taken from a small module I wrote:
And here is the declaration of ngx_command_t (the struct we’re declaring), found in core/ngx_conf_file.h :
It seems like a bit much, but each element has a purpose.
The name is the directive string, no spaces. The data type is an ngx_str_t , which is usually instantiated with just (e.g.) ngx_str("proxy_pass") . Note: an ngx_str_t is a struct with a data element, which is a string, and a len element, which is the length of that string. Nginx uses this data structure most places you’d expect a string.
type is a set of flags that indicate where the directive is legal and how many arguments the directive takes. Applicable flags, which are bitwise-OR’d, are:
- NGX_HTTP_MAIN_CONF : directive is valid in the main config
- NGX_HTTP_SRV_CONF : directive is valid in the server (host) config
- NGX_HTTP_LOC_CONF : directive is valid in a location config
- NGX_HTTP_UPS_CONF : directive is valid in an upstream config
- NGX_CONF_NOARGS : directive can take 0 arguments
- NGX_CONF_TAKE1 : directive can take exactly 1 argument
- NGX_CONF_TAKE2 : directive can take exactly 2 arguments
- NGX_CONF_TAKE7 : directive can take exactly 7 arguments
- NGX_CONF_FLAG : directive takes a boolean ("on" or "off")
- NGX_CONF_1MORE : directive must be passed at least one argument
- NGX_CONF_2MORE : directive must be passed at least two arguments
There are a few other options, too, see core/ngx_conf_file.h .
The set struct element is a pointer to a function for setting up part of the module’s configuration; typically this function will translate the arguments passed to this directive and save an appropriate value in its configuration struct. This setup function will take three arguments:
- a pointer to an ngx_conf_t struct, which contains the arguments passed to the directive
- a pointer to the current ngx_command_t struct
- a pointer to the module’s custom configuration struct
This setup function will be called when the directive is encountered. Nginx provides a number of functions for setting particular types of values in the custom configuration struct. These functions include:
- ngx_conf_set_flag_slot : translates "on" or "off" to 1 or 0
- ngx_conf_set_str_slot : saves a string as an ngx_str_t
- ngx_conf_set_num_slot : parses a number and saves it to an int
- ngx_conf_set_size_slot : parses a data size ("8k", "1m", etc.) and saves it to a size_t
There are several others, and they’re quite handy (see core/ngx_conf_file.h ). Modules can also put a reference to their own function here, if the built-ins aren’t quite good enough.
How do these built-in functions know where to save the data? That’s where the next two elements of ngx_command_t come in, conf and offset . conf tells Nginx whether this value will get saved to the module’s main configuration, server configuration, or location configuration (with NGX_HTTP_MAIN_CONF_OFFSET , NGX_HTTP_SRV_CONF_OFFSET , or NGX_HTTP_LOC_CONF_OFFSET ). offset then specifies which part of this configuration struct to write to.
The commands array is terminated with ngx_null_command as the last element.
2.3. The Module Context
This is a static ngx_http_module_t struct, which just has a bunch of function references for creating the three configurations and merging them together. Its name is ngx_http_<module name>_module_ctx . In order, the function references are:
- preconfiguration
- postconfiguration
- creating the main conf (i.e., do a malloc and set defaults)
- initializing the main conf (i.e., override the defaults with what’s in nginx.conf)
- creating the server conf
- merging it with the main conf
- creating the location conf
- merging it with the server conf
These take different arguments depending on what they’re doing. Here’s the struct definition, taken from http/ngx_http_config.h , so you can see the different function signatures of the callbacks:
You can set functions you don’t need to NULL , and Nginx will figure it out.
Most handlers just use the last two: a function to allocate memory for location-specific configuration (called ngx_http_<module name>_create_loc_conf ), and a function to set defaults and merge this configuration with any inherited configuration (called ngx_http_<module name >_merge_loc_conf ). The merge function is also responsible for producing an error if the configuration is invalid; these errors halt server startup.
Here’s an example module context struct:
Time to dig in deep a little bit. These configuration callbacks look quite similar across all modules and use the same parts of the Nginx API, so they’re worth knowing about.
2.3.1. create_loc_conf
Here’s what a bare-bones create_loc_conf function looks like, taken from the circle_gif module I wrote (see the the source ). It takes a directive struct ( ngx_conf_t ) and returns a newly created module configuration struct (in this case ngx_http_circle_gif_loc_conf_t ).
First thing to notice is Nginx’s memory allocation; it takes care of the free ’ing as long as the module uses ngx_palloc (a malloc wrapper) or ngx_pcalloc (a calloc wrapper).
The possible UNSET constants are NGX_CONF_UNSET_UINT , NGX_CONF_UNSET_PTR , NGX_CONF_UNSET_SIZE , NGX_CONF_UNSET_MSEC , and the catch-all NGX_CONF_UNSET . UNSET tell the merging function that the value should be overridden.
2.3.2. merge_loc_conf
Here’s the merging function used in the circle_gif module:
Notice first that Nginx provides nice merging functions for different data types ( ngx_conf_merge_<data type>_value ); the arguments are
- this location’s value
- the value to inherit if #1 is not set
- the default if neither #1 nor #2 is set
The result is then stored in the first argument. Available merge functions include ngx_conf_merge_size_value , ngx_conf_merge_msec_value , and others. See core/ngx_conf_file.h for a full list.
Trivia question: How do these functions write to the first argument, since the first argument is passed in by value?
Answer: these functions are defined by the preprocessor (so they expand to a few "if" statements and assignments before reaching the compiler).
Notice also how errors are produced; the function writes something to the log file, and returns NGX_CONF_ERROR . That return code halts server startup. (Since the message is logged at level NGX_LOG_EMERG , the message will also go to standard out; FYI, core/ngx_log.h has a list of log levels.)
2.4. The Module Definition
Next we add one more layer of indirection, the ngx_module_t struct. The variable is called ngx_http_<module name>_module . This is where references to the context and directives go, as well as the remaining callbacks (exit thread, exit process, etc.). The module definition is sometimes used as a key to look up data associated with a particular module. The module definition usually looks like this:
…substituting <module name> appropriately. Modules can add callbacks for process/thread creation and death, but most modules keep things simple. (For the arguments passed to each callback, see core/ngx_http_config.h .)
2.5. Module Installation
3. handlers.
Now we’ll put some trivial modules under the microscope to see how they work.
3.1. Anatomy of a Handler (Non-proxying)
Handlers typically do four things: get the location configuration, generate an appropriate response, send the header, and send the body. A handler has one argument, the request struct. A request struct has a lot of useful information about the client request, such as the request method, URI, and headers. We’ll go over these steps one by one.
3.1.1. Getting the location configuration
This part’s easy. All you need to do is call ngx_http_get_module_loc_conf and pass in the current request struct and the module definition. Here’s the relevant part of my circle gif handler:
Now I’ve got access to all the variables that I set up in my merge function.
3.1.2. Generating a response
This is the interesting part where modules actually do work.
The request struct will be helpful here, particularly these elements:
uri is the path of the request, e.g. "/query.cgi".
args is the part of the request after the question mark (e.g. "name=john").
headers_in has a lot of useful stuff, such as cookies and browser information, but many modules don’t need anything from it. See http/ngx_http_request.h if you’re interested.
This should be enough information to produce some useful output. The full ngx_http_request_t struct can be found in http/ngx_http_request.h .
3.1.3. Sending the header
The response headers live in a struct called headers_out referenced by the request struct. The handler sets the ones it wants and then calls ngx_http_send_header(r) . Some useful parts of headers_out include:
(The rest can be found in http/ngx_http_request.h .)
So for example, if a module were to set the Content-Type to "image/gif", Content-Length to 100, and return a 200 OK response, this code would do the trick:
Most legal HTTP headers are available (somewhere) for your setting pleasure. However, some headers are a bit trickier to set than the ones you see above; for example, content_encoding has type (ngx_table_elt_t*) , so the module must allocate memory for it. This is done with a function called ngx_list_push , which takes in an ngx_list_t (similar to an array) and returns a reference to a newly created member of the list (of type ngx_table_elt_t ). The following code sets the Content-Encoding to "deflate" and sends the header:
This mechanism is usually used when a header can have multiple values simultaneously; it (theoretically) makes it easier for filter modules to add and delete certain values while preserving others, because they don’t have to resort to string manipulation.
3.1.4. Sending the body
Now that the module has generated a response and put it in memory, it needs to assign the response to a special buffer, and then assign the buffer to a chain link , and then call the "send body" function on the chain link.
What are the chain links for? Nginx lets handler modules generate (and filter modules process) responses one buffer at a time; each chain link keeps a pointer to the next link in the chain, or NULL if it’s the last one. We’ll keep it simple and assume there is just one buffer.
First, a module will declare the buffer and the chain link:
The next step is to allocate the buffer and point our response data to it:
Now the module attaches it to the chain link:
FINALLY, we send the body, and return the status code of the output filter chain all in one go:
Buffer chains are a critical part of Nginx’s IO model, so you should be comfortable with how they work.
Trivia question: Why does the buffer have the last_buf variable, when we can tell we’re at the end of a chain by checking "next" for NULL ?
Answer: A chain might be incomplete, i.e., have multiple buffers, but not all the buffers in this request or response. So some buffers are at the end of the chain but not the end of a request. This brings us to…
3.2. Anatomy of an Upstream (a.k.a Proxy) Handler
I waved my hands a bit about having your handler generate a response. Sometimes you’ll be able to get that response just with a chunk of C code, but often you’ll want to talk to another server (for example, if you’re writing a module to implement another network protocol). You could do all of the network programming yourself, but what happens if you receive a partial response? You don’t want to block the primary event loop with your own event loop while you’re waiting for the rest of the response. You’d kill the Nginx’s performance. Fortunately, Nginx lets you hook right into its own mechanisms for dealing with back-end servers (called "upstreams"), so your module can talk to another server without getting in the way of other requests. This section describes how a module talks to an upstream, such as Memcached, FastCGI, or another HTTP server.
3.2.1. Summary of upstream callbacks
Unlike the handler function for other modules, the handler function of an upstream module does little "real work". It does not call ngx_http_output_filter . It merely sets callbacks that will be invoked when the upstream server is ready to be written to and read from. There are actually 6 available hooks:
create_request crafts a request buffer (or chain of them) to be sent to the upstream
reinit_request is called if the connection to the back-end is reset (just before create_request is called for the second time)
process_header processes the first bit of the upstream’s response, and usually saves a pointer to the upstream’s "payload"
abort_request is called if the client aborts the request
finalize_request is called when Nginx is finished reading from the upstream
input_filter is a body filter that can be called on the response body (e.g., to remove a trailer)
How do these get attached? An example is in order. Here’s a simplified version of the proxy module’s handler:
It does a bit of housekeeping, but the important parts are the callbacks. Also notice the bit about ngx_http_read_client_request_body . That’s setting another callback for when Nginx has finished reading from the client.
What will each of these callbacks do? Usually, reinit_request , abort_request , and finalize_request will set or reset some sort of internal state and are only a few lines long. The real workhorses are create_request and process_header .
3.2.2. The create_request callback
For the sake of simplicity, let’s suppose I have an upstream server that reads in one character and prints out two characters. What would my functions look like?
The create_request needs to allocate a buffer for the single-character request, allocate a chain link for that buffer, and then point the upstream struct to that chain link. It would look like this:
That wasn’t so bad, was it? Of course, in reality you’ll probably want to use the request URI in some meaningful way. It’s available as an ngx_str_t in r->uri , and the GET paramaters are in r->args , and don’t forget you also have access to the request headers and cookies.
3.2.3. The process_header callback
Now it’s time for the process_header . Just as create_request added a pointer to the request body, process_header shifts the response pointer to the part that the client will receive . It also reads in the header from the upstream and sets the client response headers accordingly.
Here’s a bare-minimum example, reading in that two-character response. Let’s suppose the first character is the "status" character. If it’s a question mark, we want to return a 404 File Not Found to the client and disregard the other character. If it’s a space, then we want to return the other character to the client along with a 200 OK response. All right, it’s not the most useful protocol, but it’s a good demonstration. How would we write this process_header function?
That’s it. Manipulate the header, change the pointer, it’s done. Notice that headers_in is actually a response header struct like we’ve seen before (cf. http/ngx_http_request.h ), but it can be populated with the headers from the upstream. A real proxying module will do a lot more header processing, not to mention error handling, but you get the main idea.
But… what if we don’t have the whole header from the upstream in one buffer?
3.2.4. Keeping state
Well, remember how I said that abort_request , reinit_request , and finalize_request could be used for resetting internal state? That’s because many upstream modules have internal state. The module will need to define a custom context struct to keep track of what it has read so far from an upstream. This is NOT the same as the "Module Context" referred to above. That’s of a pre-defined type, whereas the custom context can have whatever elements and data you need (it’s your struct). This context struct should be instantiated inside the create_request function, perhaps like this:
That last line essentially registers the custom context struct with a particular request and module name for easy retrieval later. Whenever you need this context struct (probably in all the other callbacks), just do:
And p will have the current state. Set it, reset it, increment, decrement, shove arbitrary data in there, whatever you want. This is a great way to use a persistent state machine when reading from an upstream that returns data in chunks, again without blocking the primary event loop. Nice!
3.3. Handler Installation
Handlers are installed by adding code to the callback of the directive that enables the module. For example, my circle gif ngx_command_t looks like this:
The callback is the third element, in this case ngx_http_circle_gif . Recall that the arguments to this callback are the directive struct ( ngx_conf_t , which holds the user’s arguments), the relevant ngx_command_t struct, and a pointer to the module’s custom configuration struct. For my circle gif module, the function looks like:
There are two steps here: first, get the "core" struct for this location, then assign a handler to it. Pretty simple, eh?
I’ve said all I know about handler modules. It’s time to move onto filter modules, the components in the output filter chain.
Filters manipulate responses generated by handlers. Header filters manipulate the HTTP headers, and body filters manipulate the response content.
4.1. Anatomy of a Header Filter
A header filter consists of three basic steps:
- Decide whether to operate on this response
- Operate on the response
- Call the next filter
To take an example, here’s a simplified version of the "not modified" header filter, which sets the status to 304 Not Modified if the client’s If-Modified-Since header matches the response’s Last-Modified header. Note that header filters take in the ngx_http_request_t struct as the only argument, which gets us access to both the client headers and soon-to-be-sent response headers.
The headers_out structure is just the same as we saw in the section about handlers (cf. http/ngx_http_request.h ), and can be manipulated to no end.
4.2. Anatomy of a Body Filter
The buffer chain makes it a little tricky to write a body filter, because the body filter can only operate on one buffer (chain link) at a time. The module must decide whether to overwrite the input buffer, replace the buffer with a newly allocated buffer, or insert a new buffer before or after the buffer in question. To complicate things, sometimes a module will receive several buffers so that it has an incomplete buffer chain that it must operate on. Unfortunately, Nginx does not provide a high-level API for manipulating the buffer chain, so body filters can be difficult to understand (and to write). But, here are some operations you might see in action.
A body filter’s prototype might look like this (example taken from the "chunked" filter in the Nginx source):
The first argument is our old friend the request struct. The second argument is a pointer to the head of the current partial chain (which could contain 0, 1, or more buffers).
Let’s take a simple example. Suppose we want to insert the text "<l!-- Served by Nginx -->" to the end of every request. First, we need to figure out if the response’s final buffer is included in the buffer chain we were given. Like I said, there’s not a fancy API, so we’ll be rolling our own for loop:
Now let’s bail out if we don’t have that last buffer:
Super, now the last buffer is stored in chain_link. Now we allocate a new buffer:
And put some data in it:
And hook the buffer into a new chain link:
Finally, hook the new chain link to the final chain link we found before:
And reset the "last_buf" variables to reflect reality:
And pass along the modified chain to the next output filter:
The resulting function takes much more effort than what you’d do with, say, mod_perl ( $response->body =~ s/$/<!-- Served by mod_perl -->/ ), but the buffer chain is a very powerful construct, allowing programmers to process data incrementally so that the client gets something as soon as possible. However, in my opinion, the buffer chain desperately needs a cleaner interface so that programmers can’t leave the chain in an inconsistent state. For now, manipulate it at your own risk.
4.3. Filter Installation
Filters are installed in the post-configuration step. We install both header filters and body filters in the same place.
Let’s take a look at the chunked filter module for a simple example. Its module context looks like this:
What’s going on here? Well, if you remember, filters are set up with a CHAIN OF RESPONSIBILITY. When a handler generates a response, it calls two functions: ngx_http_output_filter , which calls the global function reference ngx_http_top_body_filter ; and ngx_http_send_header , which calls the global function reference ngx_http_top_header_filter .
ngx_http_top_body_filter and ngx_http_top_header_filter are the respective "heads" of the body and header filter chains. Each "link" on the chain keeps a function reference to the next link in the chain (the references are called ngx_http_next_body_filter and ngx_http_next_header_filter ). When a filter is finished executing, it just calls the next filter, until a specially defined "write" filter is called, which wraps up the HTTP response. What you see in this filter_init function is the module adding itself to the filter chains; it keeps a reference to the old "top" filters in its own "next" variables and declares its functions to be the new "top" filters. (Thus, the last filter to be installed is the first to be executed.)
Side note: how does this work exactly?
Each filter either returns an error code or uses this as the return statement:
Thus, if the filter chain reaches the (specially-defined) end of the chain, an "OK" response is returned, but if there’s an error along the way, the chain is cut short and Nginx serves up the appropriate error message. It’s a singly-linked list with fast failures implemented solely with function references. Brilliant.
5. Load-Balancers
A load-balancer is just a way to decide which backend server will receive a particular request; implementations exist for distributing requests in round-robin fashion or hashing some information about the request. This section will describe both a load-balancer’s installation and its invocation, using the upstream_hash module ( full source ) as an example. upstream_hash chooses a backend by hashing a variable specified in nginx.conf.
A load-balancing module has six pieces:
- The enabling configuration directive (e.g, hash; ) will call a registration function
- resolves the server names to particular IP addresses
- allocates space for sockets
- sets a callback to the peer initialization function
- the peer initialization function, called once per request, sets up data structures that the load-balancing function will access and manipulate;
- the load-balancing function decides where to route requests; it is called at least once per client request (more, if a backend request fails). This is where the interesting stuff happens.
- and finally, the peer release function can update statistics after communication with a particular backend server has finished (whether successfully or not)
It’s a lot, but I’ll break it down into pieces.
5.1. The enabling directive
Directive declarations, recall, specify both where they’re valid and a function to call when they’re encountered. A directive for a load-balancer should have the NGX_HTTP_UPS_CONF flag set, so that Nginx knows this directive is only valid inside an upstream block. It should provide a pointer to a registration function . Here’s the directive declaration from the upstream_hash module:
Nothing new there.
5.2. The registration function
The callback ngx_http_upstream_hash above is the registration function, so named (by me) because it registers an upstream initialization function with the surrounding upstream configuration. In addition, the registration function defines which options to the server directive are legal inside this particular upstream block (e.g., weight= , fail_timeout= ). Here’s the registration function of the upstream_hash module:
Aside from jumping through hoops so we can evaluation $variable later, it’s pretty straightforward; assign a callback, set some flags. What flags are available?
- NGX_HTTP_UPSTREAM_CREATE : let there be server directives in this upstream block. I can’t think of a situation where you wouldn’t use this.
- NGX_HTTP_UPSTREAM_WEIGHT : let the server directives take a weight= option
- NGX_HTTP_UPSTREAM_MAX_FAILS : allow the max_fails= option
- NGX_HTTP_UPSTREAM_FAIL_TIMEOUT : allow the fail_timeout= option
- NGX_HTTP_UPSTREAM_DOWN : allow the down option
- NGX_HTTP_UPSTREAM_BACKUP : allow the backup option
Each module will have access to these configuration values. It’s up to the module to decide what to do with them. That is, max_fails will not be automatically enforced; all the failure logic is up to the module author. More on that later. For now, we still haven’t finished followed the trail of callbacks. Next up, we have the upstream initialization function (the init_upstream callback in the previous function).
5.3. The upstream initialization function
The purpose of the upstream initialization function is to resolve the host names, allocate space for sockets, and assign (yet another) callback. Here’s how upstream_hash does it:
This function is a bit more involved than one might hope. Most of the work seems like it should be abstracted, but it’s not, so that’s what we live with. One strategy for simplifying things is to call the upstream initialization function of another module, have it do all the dirty work (peer allocation, etc), and then override the us->peer.init callback afterwards. For an example, see http/modules/ngx_http_upstream_ip_hash_module.c .
5.4. The peer initialization function
In addition, the peer initalization function sets up two callbacks:
- get : the load-balancing function
- free : the peer release function (usually just updates some statistics when a connection finishes)
As if that weren’t enough, it also initalizes a variable called tries . As long as tries is positive, nginx will keep retrying this load-balancer. When tries is zero, nginx will give up. It’s up to the get and free functions to set tries appropriately.
Here’s a peer initialization function from the upstream_hash module:
That wasn’t so bad. Now we’re ready to pick an upstream server.
5.5. The load-balancing function
It’s time for the main course. The real meat and potatoes. This is where the module picks an upstream. The load-balancing function’s prototype looks like:
data is our struct of useful information concerning this client connection. pc will have information about the server we’re going to connect to. The job of the load-balancing function is to fill in values for pc->sockaddr , pc->socklen , and pc->name . If you know some network programming, then those variable names might be familiar; but they’re actually not very important to the task at hand. We don’t care what they stand for; we just want to know where to find appropriate values to fill them.
This function must find a list of available servers, choose one, and assign its values to pc . Let’s look at how upstream_hash does it.
upstream_hash previously stashed the server list into the ngx_http_upstream_hash_peer_data_t struct back in the call to ngx_http_upstream_init_hash (above). This struct is now available as data :
The list of peers is now stored in uhpd->peers->peer . Let’s pick a peer from this array by dividing the computed hash value by the number of servers:
Now for the grand finale:
That’s all! If the load-balancer returns NGX_OK , it means, "go ahead and try this server". If it returns NGX_BUSY , it means all the backend hosts are unavailable, and Nginx should try again.
But… how do we keep track of what’s unavailable? And what if we don’t want it to try again?
5.6. The peer release function
The peer release function operates after an upstream connection takes place; its purpose is to track failures. Here is its function prototype:
The first two parameters are just the same as we saw in the load-balancer function. The third parameter is a state variable, which indicates whether the connection was successful. It may contain two values bitwise OR’d together: NGX_PEER_FAILED (the connection failed) and NGX_PEER_NEXT (either the connection failed, or it succeeded but the application returned an error). Zero means the connection succeeded.
It’s up to the module author to decide what to do about these failure events. If they are to be used at all, the results should be stored in data , a pointer to the custom per-request data struct.
But the crucial purpose of the peer release function is to set pc->tries to zero if you don’t want Nginx to keep trying this load-balancer during this request. The simplest peer release function would look like this:
That would ensure that if there’s ever an error reaching a backend server, a 502 Bad Proxy error will be returned to the client.
Here’s a more complicated example, taken from the upstream_hash module. If a backend connection fails, it marks it as failed in a bit-vector (called tried , an array of type uintptr_t ), then keeps choosing a new backend until it finds one that has not failed.
This works because the load-balancing function will just look at the new value of uhpd->hash .
Many applications won’t need retry or high-availability logic, but it’s possible to provide it with just a few lines of code like you see here.
6. Writing and Compiling a New Nginx Module
So by now, you should be prepared to look at an Nginx module and try to understand what’s going on (and you’ll know where to look for help). Take a look in src/http/modules/ to see the available modules. Pick a module that’s similar to what you’re trying to accomplish and look through it. Stuff look familiar? It should. Refer between this guide and the module source to get an understanding about what’s going on.
First thing, you’re going to need a place to work on your module. Make a folder for your module anywhere on your hard drive, but separate from the Nginx source (and make sure you have the latest copy from nginx.net ). Your new folder should contain two files to start with:
- "ngx_http_<your module>_module.c"
The "config" file will be included by ./configure , and its contents will depend on the type of module.
"config" for filter modules:
"config" for other modules:
For more information about the "config" file format and its various options, see wiki page about it.
Now for your C file. I recommend copying an existing module that does something similar to what you want, but rename it "ngx_http_<your module>_module.c". Let this be your model as you change the behavior to suit your needs, and refer to this guide as you understand and refashion the different pieces.
When you’re ready to compile, just go into the Nginx directory and type
and then make and make install like you normally would. If all goes well, your module will be compiled right in. Nice, huh? No need to muck with the Nginx source, and adding your module to new versions of Nginx is a snap, just use that same ./configure command. By the way, if your module needs any dynamically linked libraries, you can add this to your "config" file:
Where foo is the library you need. If you make a cool or useful module, be sure to send a note to the Nginx mailing list and share your work.
7. Advanced Topics
This guide covers the basics of Nginx module development. For tips on writing more sophisticated modules, be sure to check out Emiller’s Advanced Topics In Nginx Module Development .
Appendix A: Code References
- Nginx source tree (cross-referenced)
- Nginx module directory (cross-referenced)
- Example addon: circle_gif
- Example addon: upstream_hash
- Example addon: upstream_fair
Appendix B: Changelog
- December 10, 2018: Added link documenting the "config" file format .
- August 11, 2017: Updated links to Nginx and module source code.
- January 16, 2013: Corrected code sample in 5.5.
- December 20, 2011: Corrected code sample in 4.2 (one more time).
- March 14, 2011: Corrected code sample in 4.2 (again).
- November 11, 2009: Corrected code sample in 4.2.
- August 13, 2009: Reorganized, and moved Advanced Topics to a separate article.
- July 23, 2009: Corrected code sample in 3.5.3.
- December 24, 2008: Corrected code sample in 3.4.
- July 14, 2008: Added information about subrequests; slight reorganization
- July 12, 2008: Added Grzegorz Nosek ’s guide to shared memory
- July 2, 2008: Corrected "config" file for filter modules; rewrote introduction; added TODO section
- May 28, 2007: Changed the load-balancing example to the simpler upstream_hash module
- May 19, 2007: Corrected bug in body filter example
- May 4, 2007: Added information about load-balancers
- April 28, 2007: Initial draft
Get new articles as they’re published, via LinkedIn , Twitter , or RSS .
Want to look for statistical patterns in your MySQL, PostgreSQL, or SQLite database? My desktop statistics software Wizard can help you analyze more data in less time and communicate discoveries visually without spending days struggling with pointless command syntax. Check it out!
Back to Evan Miller’s home page – Subscribe to RSS – LinkedIn – Twitter
Development guide
Introduction, code layout.
- auto — Build scripts
- core — Basic types and functions — string, array, log, pool, etc.
- modules — Event notification modules: epoll , kqueue , select etc.
- modules — Other HTTP modules
- v2 — HTTP/2
- mail — Mail modules
- stream — Stream modules
Include files
The following two #include statements must appear at the beginning of every nginx file:
#include <ngx_config.h> #include <ngx_core.h>
In addition to that, HTTP code should include
#include <ngx_http.h>
Mail code should include
#include <ngx_mail.h>
Stream code should include
#include <ngx_stream.h>
For general purposes, nginx code uses two integer types, ngx_int_t and ngx_uint_t , which are typedefs for intptr_t and uintptr_t respectively.
Common return codes
Most functions in nginx return the following codes:
- NGX_OK — Operation succeeded.
- NGX_ERROR — Operation failed.
- NGX_AGAIN — Operation incomplete; call the function again.
- NGX_DECLINED — Operation rejected, for example, because it is disabled in the configuration. This is never an error.
- NGX_BUSY — Resource is not available.
- NGX_DONE — Operation complete or continued elsewhere. Also used as an alternative success code.
- NGX_ABORT — Function was aborted. Also used as an alternative error code.
Error handling
The ngx_errno macro returns the last system error code. It's mapped to errno on POSIX platforms and to GetLastError() call in Windows. The ngx_socket_errno macro returns the last socket error number. Like the ngx_errno macro, it's mapped to errno on POSIX platforms. It's mapped to the WSAGetLastError() call on Windows. Accessing the values of ngx_errno or ngx_socket_errno more than once in a row can cause performance issues. If the error value might be used multiple times, store it in a local variable of type ngx_err_t . To set errors, use the ngx_set_errno(errno) and ngx_set_socket_errno(errno) macros.
The values of ngx_errno and ngx_socket_errno can be passed to the logging functions ngx_log_error() and ngx_log_debugX() , in which case system error text is added to the log message.
Example using ngx_errno :
ngx_int_t ngx_my_kill(ngx_pid_t pid, ngx_log_t *log, int signo) { ngx_err_t err; if (kill(pid, signo) == -1) { err = ngx_errno; ngx_log_error(NGX_LOG_ALERT, log, err, "kill(%P, %d) failed", pid, signo); if (err == NGX_ESRCH) { return 2; } return 1; } return 0; }
For C strings, nginx uses the unsigned character type pointer u_char * .
The nginx string type ngx_str_t is defined as follows:
typedef struct { size_t len; u_char *data; } ngx_str_t;
The len field holds the string length and data holds the string data. The string, held in ngx_str_t , may or may not be null-terminated after the len bytes. In most cases it’s not. However, in certain parts of the code (for example, when parsing configuration), ngx_str_t objects are known to be null-terminated, which simplifies string comparison and makes it easier to pass the strings to syscalls.
The string operations in nginx are declared in src/core/ngx_string.h Some of them are wrappers around standard C functions:
- ngx_strcmp()
- ngx_strncmp()
- ngx_strstr()
- ngx_strlen()
- ngx_strchr()
- ngx_memcmp()
- ngx_memset()
- ngx_memcpy()
- ngx_memmove()
Other string functions are nginx-specific
- ngx_memzero() — Fills memory with zeroes.
- ngx_explicit_memzero() — Does the same as ngx_memzero() , but this call is never removed by the compiler's dead store elimination optimization. This function can be used to clear sensitive data such as passwords and keys.
- ngx_cpymem() — Does the same as ngx_memcpy() , but returns the final destination address This one is handy for appending multiple strings in a row.
- ngx_movemem() — Does the same as ngx_memmove() , but returns the final destination address.
- ngx_strlchr() — Searches for a character in a string, delimited by two pointers.
The following functions perform case conversion and comparison:
- ngx_tolower()
- ngx_toupper()
- ngx_strlow()
- ngx_strcasecmp()
- ngx_strncasecmp()
The following macros simplify string initialization:
- ngx_string(text) — static initializer for the ngx_str_t type from the C string literal text
- ngx_null_string — static empty string initializer for the ngx_str_t type
- ngx_str_set(str, text) — initializes string str of ngx_str_t * type with the C string literal text
- ngx_str_null(str) — initializes string str of ngx_str_t * type with the empty string
The following formatting functions support nginx-specific types:
- ngx_sprintf(buf, fmt, ...)
- ngx_snprintf(buf, max, fmt, ...)
- ngx_slprintf(buf, last, fmt, ...)
- ngx_vslprintf(buf, last, fmt, args)
- ngx_vsnprintf(buf, max, fmt, args)
The full list of formatting options, supported by these functions is in src/core/ngx_string.c . Some of them are:
- %T — time_t
- %z — ssize_t
- %i — ngx_int_t
- %p — void *
- %V — ngx_str_t *
- %s — u_char * (null-terminated)
- %*s — size_t + u_char *
You can prepend u on most types to make them unsigned. To convert output to hex, use X or x .
For example:
u_char buf[NGX_INT_T_LEN]; size_t len; ngx_uint_t n; /* set n here */ len = ngx_sprintf(buf, "%ui", n) — buf;
Numeric conversion
Several functions for numeric conversion are implemented in nginx. The first four each convert a string of given length to a positive integer of the indicated type. They return NGX_ERROR on error.
- ngx_atoi(line, n) — ngx_int_t
- ngx_atosz(line, n) — ssize_t
- ngx_atoof(line, n) — off_t
- ngx_atotm(line, n) — time_t
There are two additional numeric conversion functions. Like the first four, they return NGX_ERROR on error.
- ngx_atofp(line, n, point) — Converts a fixed point floating number of given length to a positive integer of type ngx_int_t . The result is shifted left by point decimal positions. The string representation of the number is expected to have no more than points fractional digits. For example, ngx_atofp("10.5", 4, 2) returns 1050 .
- ngx_hextoi(line, n) — Converts a hexadecimal representation of a positive integer to ngx_int_t .
Regular expressions
The regular expressions interface in nginx is a wrapper around the PCRE library. The corresponding header file is src/core/ngx_regex.h .
To use a regular expression for string matching, it first needs to be compiled, which is usually done at the configuration phase. Note that since PCRE support is optional, all code using the interface must be protected by the surrounding NGX_PCRE macro:
#if (NGX_PCRE) ngx_regex_t *re; ngx_regex_compile_t rc; u_char errstr[NGX_MAX_CONF_ERRSTR]; ngx_str_t value = ngx_string("message (\\d\\d\\d).*Codeword is '(?<cw>\\w+)'"); ngx_memzero(&rc, sizeof(ngx_regex_compile_t)); rc.pattern = value; rc.pool = cf->pool; rc.err.len = NGX_MAX_CONF_ERRSTR; rc.err.data = errstr; /* rc.options can be set to NGX_REGEX_CASELESS */ if (ngx_regex_compile(&rc) != NGX_OK) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "%V", &rc.err); return NGX_CONF_ERROR; } re = rc.regex; #endif
After successful compilation, the captures and named_captures fields in the ngx_regex_compile_t structure contain the count of all captures and named captures, respectively, found in the regular expression.
The compiled regular expression can then be used for matching against strings:
ngx_int_t n; int captures[(1 + rc.captures) * 3]; ngx_str_t input = ngx_string("This is message 123. Codeword is 'foobar'."); n = ngx_regex_exec(re, &input, captures, (1 + rc.captures) * 3); if (n >= 0) { /* string matches expression */ } else if (n == NGX_REGEX_NO_MATCHED) { /* no match was found */ } else { /* some error */ ngx_log_error(NGX_LOG_ALERT, log, 0, ngx_regex_exec_n " failed: %i", n); }
The arguments to ngx_regex_exec() are the compiled regular expression re , the string to match input , an optional array of integers to hold any captures that are found, and the array's size . The size of the captures array must be a multiple of three, as required by the PCRE API . In the example, the size is calculated from the total number of captures plus one for the matched string itself.
If there are matches, captures can be accessed as follows:
u_char *p; size_t size; ngx_str_t name, value; /* all captures */ for (i = 0; i < n * 2; i += 2) { value.data = input.data + captures[i]; value.len = captures[i + 1] — captures[i]; } /* accessing named captures */ size = rc.name_size; p = rc.names; for (i = 0; i < rc.named_captures; i++, p += size) { /* capture name */ name.data = &p[2]; name.len = ngx_strlen(name.data); n = 2 * ((p[0] << 8) + p[1]); /* captured value */ value.data = &input.data[captures[n]]; value.len = captures[n + 1] — captures[n]; }
The ngx_regex_exec_array() function accepts the array of ngx_regex_elt_t elements (which are just compiled regular expressions with associated names), a string to match, and a log. The function applies expressions from the array to the string until either a match is found or no more expressions are left. The return value is NGX_OK when there is a match and NGX_DECLINED otherwise, or NGX_ERROR in case of error.
The ngx_time_t structure represents time with three separate types for seconds, milliseconds, and the GMT offset:
typedef struct { time_t sec; ngx_uint_t msec; ngx_int_t gmtoff; } ngx_time_t;
The ngx_tm_t structure is an alias for struct tm on UNIX platforms and SYSTEMTIME on Windows.
To obtain the current time, it is usually sufficient to access one of the available global variables, representing the cached time value in the desired format.
The available string representations are:
- ngx_cached_err_log_time — Used in error log entries: "1970/09/28 12:00:00"
- ngx_cached_http_log_time — Used in HTTP access log entries: "28/Sep/1970:12:00:00 +0600"
- ngx_cached_syslog_time — Used in syslog entries: "Sep 28 12:00:00"
- ngx_cached_http_time — Used in HTTP headers: "Mon, 28 Sep 1970 06:00:00 GMT"
- ngx_cached_http_log_iso8601 — The ISO 8601 standard format: "1970-09-28T12:00:00+06:00"
The ngx_time() and ngx_timeofday() macros return the current time value in seconds and are the preferred way to access the cached time value.
To obtain the time explicitly, use ngx_gettimeofday() , which updates its argument (pointer to struct timeval ). The time is always updated when nginx returns to the event loop from system calls. To update the time immediately, call ngx_time_update() , or ngx_time_sigsafe_update() if updating the time in the signal handler context.
The following functions convert time_t into the indicated broken-down time representation. The first function in each pair converts time_t to ngx_tm_t and the second (with the _libc_ infix) to struct tm :
- ngx_gmtime(), ngx_libc_gmtime() — Time expressed as UTC
- ngx_localtime(), ngx_libc_localtime() — Time expressed relative to the local time zone
The ngx_http_time(buf, time) function returns a string representation suitable for use in HTTP headers (for example, "Mon, 28 Sep 1970 06:00:00 GMT" ). The ngx_http_cookie_time(buf, time) returns a string representation function returns a string representation suitable for HTTP cookies ( "Thu, 31-Dec-37 23:55:55 GMT" ).
The nginx array type ngx_array_t is defined as follows
typedef struct { void *elts; ngx_uint_t nelts; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_array_t;
The elements of the array are available in the elts field. The nelts field holds the number of elements. The size field holds the size of a single element and is set when the array is initialized.
Use the ngx_array_create(pool, n, size) call to create an array in a pool, and the ngx_array_init(array, pool, n, size) call to initialize an array object that has already been allocated.
ngx_array_t *a, b; /* create an array of strings with preallocated memory for 10 elements */ a = ngx_array_create(pool, 10, sizeof(ngx_str_t)); /* initialize string array for 10 elements */ ngx_array_init(&b, pool, 10, sizeof(ngx_str_t));
Use the following functions to add elements to an array:
- ngx_array_push(a) adds one tail element and returns pointer to it
- ngx_array_push_n(a, n) adds n tail elements and returns pointer to the first one
If the currently allocated amount of memory is not large enough to accommodate the new elements, a new block of memory is allocated and the existing elements are copied to it. The new memory block is normally twice as large as the existing one.
s = ngx_array_push(a); ss = ngx_array_push_n(&b, 3);
In nginx a list is a sequence of arrays, optimized for inserting a potentially large number of items. The ngx_list_t list type is defined as follows:
typedef struct { ngx_list_part_t *last; ngx_list_part_t part; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_list_t;
The actual items are stored in list parts, which are defined as follows:
typedef struct ngx_list_part_s ngx_list_part_t; struct ngx_list_part_s { void *elts; ngx_uint_t nelts; ngx_list_part_t *next; };
Before use, a list must be initialized by calling ngx_list_init(list, pool, n, size) or created by calling ngx_list_create(pool, n, size) . Both functions take as arguments the size of a single item and a number of items per list part. To add an item to a list, use the ngx_list_push(list) function. To iterate over the items, directly access the list fields as shown in the example:
ngx_str_t *v; ngx_uint_t i; ngx_list_t *list; ngx_list_part_t *part; list = ngx_list_create(pool, 100, sizeof(ngx_str_t)); if (list == NULL) { /* error */ } /* add items to the list */ v = ngx_list_push(list); if (v == NULL) { /* error */ } ngx_str_set(v, "foo"); v = ngx_list_push(list); if (v == NULL) { /* error */ } ngx_str_set(v, "bar"); /* iterate over the list */ part = &list->part; v = part->elts; for (i = 0; /* void */; i++) { if (i >= part->nelts) { if (part->next == NULL) { break; } part = part->next; v = part->elts; i = 0; } ngx_do_smth(&v[i]); }
Lists are primarily used for HTTP input and output headers.
Lists do not support item removal. However, when needed, items can internally be marked as missing without actually being removed from the list. For example, to mark HTTP output headers (which are stored as ngx_table_elt_t objects) as missing, set the hash field in ngx_table_elt_t to zero. Items marked in this way are explicitly skipped when the headers are iterated over.
In nginx a queue is an intrusive doubly linked list, with each node defined as follows:
typedef struct ngx_queue_s ngx_queue_t; struct ngx_queue_s { ngx_queue_t *prev; ngx_queue_t *next; };
The head queue node is not linked with any data. Use the ngx_queue_init(q) call to initialize the list head before use. Queues support the following operations:
- ngx_queue_insert_head(h, x) , ngx_queue_insert_tail(h, x) — Insert a new node
- ngx_queue_remove(x) — Remove a queue node
- ngx_queue_split(h, q, n) — Split a queue at a node, returning the queue tail in a separate queue
- ngx_queue_add(h, n) — Add a second queue to the first queue
- ngx_queue_head(h) , ngx_queue_last(h) — Get first or last queue node
- ngx_queue_sentinel(h) - Get a queue sentinel object to end iteration at
- ngx_queue_data(q, type, link) — Get a reference to the beginning of a queue node data structure, considering the queue field offset in it
An example:
typedef struct { ngx_str_t value; ngx_queue_t queue; } ngx_foo_t; ngx_foo_t *f; ngx_queue_t values, *q; ngx_queue_init(&values); f = ngx_palloc(pool, sizeof(ngx_foo_t)); if (f == NULL) { /* error */ } ngx_str_set(&f->value, "foo"); ngx_queue_insert_tail(&values, &f->queue); /* insert more nodes here */ for (q = ngx_queue_head(&values); q != ngx_queue_sentinel(&values); q = ngx_queue_next(q)) { f = ngx_queue_data(q, ngx_foo_t, queue); ngx_do_smth(&f->value); }
Red-Black tree
The src/core/ngx_rbtree.h header file provides access to the effective implementation of red-black trees.
typedef struct { ngx_rbtree_t rbtree; ngx_rbtree_node_t sentinel; /* custom per-tree data here */ } my_tree_t; typedef struct { ngx_rbtree_node_t rbnode; /* custom per-node data */ foo_t val; } my_node_t;
To deal with a tree as a whole, you need two nodes: root and sentinel. Typically, they are added to a custom structure, allowing you to organize your data into a tree in which the leaves contain a link to or embed your data.
To initialize a tree:
my_tree_t root; ngx_rbtree_init(&root.rbtree, &root.sentinel, insert_value_function);
To traverse a tree and insert new values, use the " insert_value " functions. For example, the ngx_str_rbtree_insert_value function deals with the ngx_str_t type. Its arguments are pointers to a root node of an insertion, the newly created node to be added, and a tree sentinel.
void ngx_str_rbtree_insert_value(ngx_rbtree_node_t *temp, ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel)
The traversal is pretty straightforward and can be demonstrated with the following lookup function pattern:
my_node_t * my_rbtree_lookup(ngx_rbtree_t *rbtree, foo_t *val, uint32_t hash) { ngx_int_t rc; my_node_t *n; ngx_rbtree_node_t *node, *sentinel; node = rbtree->root; sentinel = rbtree->sentinel; while (node != sentinel) { n = (my_node_t *) node; if (hash != node->key) { node = (hash < node->key) ? node->left : node->right; continue; } rc = compare(val, node->val); if (rc < 0) { node = node->left; continue; } if (rc > 0) { node = node->right; continue; } return n; } return NULL; }
The compare() function is a classic comparator function that returns a value less than, equal to, or greater than zero. To speed up lookups and avoid comparing user objects that can be big, an integer hash field is used.
To add a node to a tree, allocate a new node, initialize it and call ngx_rbtree_insert() :
my_node_t *my_node; ngx_rbtree_node_t *node; my_node = ngx_palloc(...); init_custom_data(&my_node->val); node = &my_node->rbnode; node->key = create_key(my_node->val); ngx_rbtree_insert(&root->rbtree, node);
To remove a node, call the ngx_rbtree_delete() function:
ngx_rbtree_delete(&root->rbtree, node);
Hash table functions are declared in src/core/ngx_hash.h . Both exact and wildcard matching are supported. The latter requires extra setup and is described in a separate section below.
Before initializing a hash, you need to know the number of elements it will hold so that nginx can build it optimally. Two parameters that need to be configured are max_size and bucket_size , as detailed in a separate document . They are usually configurable by the user. Hash initialization settings are stored with the ngx_hash_init_t type, and the hash itself is ngx_hash_t :
ngx_hash_t foo_hash; ngx_hash_init_t hash; hash.hash = &foo_hash; hash.key = ngx_hash_key; hash.max_size = 512; hash.bucket_size = ngx_align(64, ngx_cacheline_size); hash.name = "foo_hash"; hash.pool = cf->pool; hash.temp_pool = cf->temp_pool;
The key is a pointer to a function that creates the hash integer key from a string. There are two generic key-creation functions: ngx_hash_key(data, len) and ngx_hash_key_lc(data, len) . The latter converts a string to all lowercase characters, so the passed string must be writable. If that is not true, pass the NGX_HASH_READONLY_KEY flag to the function, initializing the key array (see below).
The hash keys are stored in ngx_hash_keys_arrays_t and are initialized with ngx_hash_keys_array_init(arr, type) : The second parameter ( type ) controls the amount of resources preallocated for the hash and can be either NGX_HASH_SMALL or NGX_HASH_LARGE . The latter is appropriate if you expect the hash to contain thousands of elements.
ngx_hash_keys_arrays_t foo_keys; foo_keys.pool = cf->pool; foo_keys.temp_pool = cf->temp_pool; ngx_hash_keys_array_init(&foo_keys, NGX_HASH_SMALL);
To insert keys into a hash keys array, use the ngx_hash_add_key(keys_array, key, value, flags) function:
ngx_str_t k1 = ngx_string("key1"); ngx_str_t k2 = ngx_string("key2"); ngx_hash_add_key(&foo_keys, &k1, &my_data_ptr_1, NGX_HASH_READONLY_KEY); ngx_hash_add_key(&foo_keys, &k2, &my_data_ptr_2, NGX_HASH_READONLY_KEY);
To build the hash table, call the ngx_hash_init(hinit, key_names, nelts) function:
ngx_hash_init(&hash, foo_keys.keys.elts, foo_keys.keys.nelts);
The function fails if max_size or bucket_size parameters are not big enough.
When the hash is built, use the ngx_hash_find(hash, key, name, len) function to look up elements:
my_data_t *data; ngx_uint_t key; key = ngx_hash_key(k1.data, k1.len); data = ngx_hash_find(&foo_hash, key, k1.data, k1.len); if (data == NULL) { /* key not found */ }
Wildcard matching
To create a hash that works with wildcards, use the ngx_hash_combined_t type. It includes the hash type described above and has two additional keys arrays: dns_wc_head and dns_wc_tail . The initialization of basic properties is similar to a regular hash:
ngx_hash_init_t hash ngx_hash_combined_t foo_hash; hash.hash = &foo_hash.hash; hash.key = ...;
It is possible to add wildcard keys using the NGX_HASH_WILDCARD_KEY flag:
/* k1 = ".example.org"; */ /* k2 = "foo.*"; */ ngx_hash_add_key(&foo_keys, &k1, &data1, NGX_HASH_WILDCARD_KEY); ngx_hash_add_key(&foo_keys, &k2, &data2, NGX_HASH_WILDCARD_KEY);
The function recognizes wildcards and adds keys into the corresponding arrays. Please refer to the map module documentation for the description of the wildcard syntax and the matching algorithm.
Depending on the contents of added keys, you may need to initialize up to three key arrays: one for exact matching (described above), and two more to enable matching starting from the head or tail of a string:
if (foo_keys.dns_wc_head.nelts) { ngx_qsort(foo_keys.dns_wc_head.elts, (size_t) foo_keys.dns_wc_head.nelts, sizeof(ngx_hash_key_t), cmp_dns_wildcards); hash.hash = NULL; hash.temp_pool = pool; if (ngx_hash_wildcard_init(&hash, foo_keys.dns_wc_head.elts, foo_keys.dns_wc_head.nelts) != NGX_OK) { return NGX_ERROR; } foo_hash.wc_head = (ngx_hash_wildcard_t *) hash.hash; }
The keys array needs to be sorted, and initialization results must be added to the combined hash. The initialization of dns_wc_tail array is done similarly.
The lookup in a combined hash is handled by the ngx_hash_find_combined(chash, key, name, len) :
/* key = "bar.example.org"; — will match ".example.org" */ /* key = "foo.example.com"; — will match "foo.*" */ hkey = ngx_hash_key(key.data, key.len); res = ngx_hash_find_combined(&foo_hash, hkey, key.data, key.len);
Memory management
To allocate memory from system heap, use the following functions:
- ngx_alloc(size, log) — Allocate memory from system heap. This is a wrapper around malloc() with logging support. Allocation error and debugging information is logged to log .
- ngx_calloc(size, log) — Allocate memory from system heap like ngx_alloc() , but fill memory with zeros after allocation.
- ngx_memalign(alignment, size, log) — Allocate aligned memory from system heap. This is a wrapper around posix_memalign() on those platforms that provide that function. Otherwise implementation falls back to ngx_alloc() which provides maximum alignment.
- ngx_free(p) — Free allocated memory. This is a wrapper around free()
Most nginx allocations are done in pools. Memory allocated in an nginx pool is freed automatically when the pool is destroyed. This provides good allocation performance and makes memory control easy.
A pool internally allocates objects in continuous blocks of memory. Once a block is full, a new one is allocated and added to the pool memory block list. When the requested allocation is too large to fit into a block, the request is forwarded to the system allocator and the returned pointer is stored in the pool for further deallocation.
The type for nginx pools is ngx_pool_t . The following operations are supported:
- ngx_create_pool(size, log) — Create a pool with specified block size. The pool object returned is allocated in the pool as well. The size should be at least NGX_MIN_POOL_SIZE and a multiple of NGX_POOL_ALIGNMENT .
- ngx_destroy_pool(pool) — Free all pool memory, including the pool object itself.
- ngx_palloc(pool, size) — Allocate aligned memory from the specified pool.
- ngx_pcalloc(pool, size) — Allocate aligned memory from the specified pool and fill it with zeroes.
- ngx_pnalloc(pool, size) — Allocate unaligned memory from the specified pool. Mostly used for allocating strings.
- ngx_pfree(pool, p) — Free memory that was previously allocated in the specified pool. Only allocations that result from requests forwarded to the system allocator can be freed.
u_char *p; ngx_str_t *s; ngx_pool_t *pool; pool = ngx_create_pool(1024, log); if (pool == NULL) { /* error */ } s = ngx_palloc(pool, sizeof(ngx_str_t)); if (s == NULL) { /* error */ } ngx_str_set(s, "foo"); p = ngx_pnalloc(pool, 3); if (p == NULL) { /* error */ } ngx_memcpy(p, "foo", 3);
Chain links ( ngx_chain_t ) are actively used in nginx, so the nginx pool implementation provides a way to reuse them. The chain field of ngx_pool_t keeps a list of previously allocated links ready for reuse. For efficient allocation of a chain link in a pool, use the ngx_alloc_chain_link(pool) function. This function looks up a free chain link in the pool list and allocates a new chain link if the pool list is empty. To free a link, call the ngx_free_chain(pool, cl) function.
Cleanup handlers can be registered in a pool. A cleanup handler is a callback with an argument which is called when pool is destroyed. A pool is usually tied to a specific nginx object (like an HTTP request) and is destroyed when the object reaches the end of its lifetime. Registering a pool cleanup is a convenient way to release resources, close file descriptors or make final adjustments to the shared data associated with the main object.
To register a pool cleanup, call ngx_pool_cleanup_add(pool, size) , which returns a ngx_pool_cleanup_t pointer to be filled in by the caller. Use the size argument to allocate context for the cleanup handler.
ngx_pool_cleanup_t *cln; cln = ngx_pool_cleanup_add(pool, 0); if (cln == NULL) { /* error */ } cln->handler = ngx_my_cleanup; cln->data = "foo"; ... static void ngx_my_cleanup(void *data) { u_char *msg = data; ngx_do_smth(msg); }
Shared memory
Shared memory is used by nginx to share common data between processes. The ngx_shared_memory_add(cf, name, size, tag) function adds a new shared memory entry ngx_shm_zone_t to a cycle. The function receives the name and size of the zone. Each shared zone must have a unique name. If a shared zone entry with the provided name and tag already exists, the existing zone entry is reused. The function fails with an error if an existing entry with the same name has a different tag. Usually, the address of the module structure is passed as tag , making it possible to reuse shared zones by name within one nginx module.
The shared memory entry structure ngx_shm_zone_t has the following fields:
- init — Initialization callback, called after the shared zone is mapped to actual memory
- data — Data context, used to pass arbitrary data to the init callback
- noreuse — Flag that disables reuse of a shared zone from the old cycle
- tag — Shared zone tag
- addr — Mapped shared memory address, initially NULL
- size — Shared memory size
- name — Shared memory name
- log — Shared memory log
- exists — Flag that indicates shared memory was inherited from the master process (Windows-specific)
Shared zone entries are mapped to actual memory in ngx_init_cycle() after the configuration is parsed. On POSIX systems, the mmap() syscall is used to create the shared anonymous mapping. On Windows, the CreateFileMapping() / MapViewOfFileEx() pair is used.
For allocating in shared memory, nginx provides the slab pool ngx_slab_pool_t type. A slab pool for allocating memory is automatically created in each nginx shared zone. The pool is located in the beginning of the shared zone and can be accessed by the expression (ngx_slab_pool_t *) shm_zone->shm.addr . To allocate memory in a shared zone, call either ngx_slab_alloc(pool, size) or ngx_slab_calloc(pool, size) . To free memory, call ngx_slab_free(pool, p) .
Slab pool divides all shared zone into pages. Each page is used for allocating objects of the same size. The specified size must be a power of 2, and greater than the minimum size of 8 bytes. Nonconforming values are rounded up. A bitmask for each page tracks which blocks are in use and which are free for allocation. For sizes greater than a half page (which is usually 2048 bytes), allocation is done an entire page at a time
To protect data in shared memory from concurrent access, use the mutex available in the mutex field of ngx_slab_pool_t . A mutex is most commonly used by the slab pool while allocating and freeing memory, but it can be used to protect any other user data structures allocated in the shared zone. To lock or unlock a mutex, call ngx_shmtx_lock(&shpool->mutex) or ngx_shmtx_unlock(&shpool->mutex) respectively.
ngx_str_t name; ngx_foo_ctx_t *ctx; ngx_shm_zone_t *shm_zone; ngx_str_set(&name, "foo"); /* allocate shared zone context */ ctx = ngx_pcalloc(cf->pool, sizeof(ngx_foo_ctx_t)); if (ctx == NULL) { /* error */ } /* add an entry for 64k shared zone */ shm_zone = ngx_shared_memory_add(cf, &name, 65536, &ngx_foo_module); if (shm_zone == NULL) { /* error */ } /* register init callback and context */ shm_zone->init = ngx_foo_init_zone; shm_zone->data = ctx; ... static ngx_int_t ngx_foo_init_zone(ngx_shm_zone_t *shm_zone, void *data) { ngx_foo_ctx_t *octx = data; size_t len; ngx_foo_ctx_t *ctx; ngx_slab_pool_t *shpool; value = shm_zone->data; if (octx) { /* reusing a shared zone from old cycle */ ctx->value = octx->value; return NGX_OK; } shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; if (shm_zone->shm.exists) { /* initialize shared zone context in Windows nginx worker */ ctx->value = shpool->data; return NGX_OK; } /* initialize shared zone */ ctx->value = ngx_slab_alloc(shpool, sizeof(ngx_uint_t)); if (ctx->value == NULL) { return NGX_ERROR; } shpool->data = ctx->value; return NGX_OK; }
For logging nginx uses ngx_log_t objects. The nginx logger supports several types of output:
- stderr — Logging to standard error (stderr)
- file — Logging to a file
- syslog — Logging to syslog
- memory — Logging to internal memory storage for development purposes; the memory can be accessed later with a debugger
A logger instance can be a chain of loggers, linked to each other with the next field. In this case, each message is written to all loggers in the chain.
For each logger, a severity level controls which messages are written to the log (only events assigned that level or higher are logged). The following severity levels are supported:
- NGX_LOG_EMERG
- NGX_LOG_ALERT
- NGX_LOG_CRIT
- NGX_LOG_ERR
- NGX_LOG_WARN
- NGX_LOG_NOTICE
- NGX_LOG_INFO
- NGX_LOG_DEBUG
For debug logging, the debug mask is checked as well. The debug masks are:
- NGX_LOG_DEBUG_CORE
- NGX_LOG_DEBUG_ALLOC
- NGX_LOG_DEBUG_MUTEX
- NGX_LOG_DEBUG_EVENT
- NGX_LOG_DEBUG_HTTP
- NGX_LOG_DEBUG_MAIL
- NGX_LOG_DEBUG_STREAM
Normally, loggers are created by existing nginx code from error_log directives and are available at nearly every stage of processing in cycle, configuration, client connection and other objects.
Nginx provides the following logging macros:
- ngx_log_error(level, log, err, fmt, ...) — Error logging
- ngx_log_debug0(level, log, err, fmt) , ngx_log_debug1(level, log, err, fmt, arg1) etc — Debug logging with up to eight supported formatting arguments
A log message is formatted in a buffer of size NGX_MAX_ERROR_STR (currently, 2048 bytes) on stack. The message is prepended with the severity level, process ID (PID), connection ID (stored in log->connection ), and the system error text. For non-debug messages log->handler is called as well to prepend more specific information to the log message. HTTP module sets ngx_http_log_error() function as log handler to log client and server addresses, current action (stored in log->action ), client request line, server name etc.
/* specify what is currently done */ log->action = "sending mp4 to client"; /* error and debug log */ ngx_log_error(NGX_LOG_INFO, c->log, 0, "client prematurely closed connection"); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "mp4 start:%ui, length:%ui", mp4->start, mp4->length);
The example above results in log entries like these:
2016/09/16 22:08:52 [info] 17445#0: *1 client prematurely closed connection while sending mp4 to client, client: 127.0.0.1, server: , request: "GET /file.mp4 HTTP/1.1" 2016/09/16 23:28:33 [debug] 22140#0: *1 mp4 start:0, length:10000
A cycle object stores the nginx runtime context created from a specific configuration. Its type is ngx_cycle_t . The current cycle is referenced by the ngx_cycle global variable and inherited by nginx workers as they start. Each time the nginx configuration is reloaded, a new cycle is created from the new nginx configuration; the old cycle is usually deleted after the new one is successfully created.
A cycle is created by the ngx_init_cycle() function, which takes the previous cycle as its argument. The function locates the previous cycle's configuration file and inherits as many resources as possible from the previous cycle. A placeholder cycle called "init cycle" is created as nginx start, then is replaced by an actual cycle built from configuration.
Members of the cycle include:
- pool — Cycle pool. Created for each new cycle.
- log — Cycle log. Initially inherited from the old cycle, it is set to point to new_log after the configuration is read.
- new_log — Cycle log, created by the configuration. It's affected by the root-scope error_log directive.
- connections , connection_n — Array of connections of type ngx_connection_t , created by the event module while initializing each nginx worker. The worker_connections directive in the nginx configuration sets the number of connections connection_n .
- free_connections , free_connection_n — List and number of currently available connections. If no connections are available, an nginx worker refuses to accept new clients or connect to upstream servers.
- files , files_n — Array for mapping file descriptors to nginx connections. This mapping is used by the event modules, having the NGX_USE_FD_EVENT flag (currently, it's poll and devpoll ).
- conf_ctx — Array of core module configurations. The configurations are created and filled during reading of nginx configuration files.
- modules , modules_n — Array of modules of type ngx_module_t , both static and dynamic, loaded by the current configuration.
- listening — Array of listening objects of type ngx_listening_t . Listening objects are normally added by the listen directive of different modules which call the ngx_create_listening() function. Listen sockets are created based on the listening objects.
- path loader — Executes only once in 60 seconds after starting or reloading nginx. Normally, the loader reads the directory and stores data in nginx shared memory. The handler is called from the dedicated nginx process “nginx cache loader”.
- path manager — Executes periodically. Normally, the manager removes old files from the directory and updates nginx memory to reflect the changes. The handler is called from the dedicated “nginx cache manager” process.
- open_files — List of open file objects of type ngx_open_file_t , which are created by calling the function ngx_conf_open_file() . Currently, nginx uses this kind of open files for logging. After reading the configuration, nginx opens all files in the open_files list and stores each file descriptor in the object's fd field. The files are opened in append mode and are created if missing. The files in the list are reopened by nginx workers upon receiving the reopen signal (most often USR1 ). In this case the descriptor in the fd field is changed to a new value.
- shared_memory — List of shared memory zones, each added by calling the ngx_shared_memory_add() function. Shared zones are mapped to the same address range in all nginx processes and are used to share common data, for example the HTTP cache in-memory tree.
For input/output operations, nginx provides the buffer type ngx_buf_t . Normally, it's used to hold data to be written to a destination or read from a source. A buffer can reference data in memory or in a file and it's technically possible for a buffer to reference both at the same time. Memory for the buffer is allocated separately and is not related to the buffer structure ngx_buf_t .
The ngx_buf_t structure has the following fields:
- start , end — The boundaries of the memory block allocated for the buffer.
- pos , last — The boundaries of the memory buffer; normally a subrange of start .. end .
- file_pos , file_last — The boundaries of a file buffer, expressed as offsets from the beginning of the file.
- tag — Unique value used to distinguish buffers; created by different nginx modules, usually for the purpose of buffer reuse.
- file — File object.
- temporary — Flag indicating that the buffer references writable memory.
- memory — Flag indicating that the buffer references read-only memory.
- in_file — Flag indicating that the buffer references data in a file.
- flush — Flag indicating that all data prior to the buffer need to be flushed.
- recycled — Flag indicating that the buffer can be reused and needs to be consumed as soon as possible.
- sync — Flag indicating that the buffer carries no data or special signal like flush or last_buf . By default nginx considers such buffers an error condition, but this flag tells nginx to skip the error check.
- last_buf — Flag indicating that the buffer is the last in output.
- last_in_chain — Flag indicating that there are no more data buffers in a request or subrequest.
- shadow — Reference to another ("shadow") buffer related to the current buffer, usually in the sense that the buffer uses data from the shadow. When the buffer is consumed, the shadow buffer is normally also marked as consumed.
- last_shadow — Flag indicating that the buffer is the last one that references a particular shadow buffer.
- temp_file — Flag indicating that the buffer is in a temporary file.
For input and output operations buffers are linked in chains. A chain is a sequence of chain links of type ngx_chain_t , defined as follows:
typedef struct ngx_chain_s ngx_chain_t; struct ngx_chain_s { ngx_buf_t *buf; ngx_chain_t *next; };
Each chain link keeps a reference to its buffer and a reference to the next chain link.
An example of using buffers and chains:
ngx_chain_t * ngx_get_my_chain(ngx_pool_t *pool) { ngx_buf_t *b; ngx_chain_t *out, *cl, **ll; /* first buf */ cl = ngx_alloc_chain_link(pool); if (cl == NULL) { /* error */ } b = ngx_calloc_buf(pool); if (b == NULL) { /* error */ } b->start = (u_char *) "foo"; b->pos = b->start; b->end = b->start + 3; b->last = b->end; b->memory = 1; /* read-only memory */ cl->buf = b; out = cl; ll = &cl->next; /* second buf */ cl = ngx_alloc_chain_link(pool); if (cl == NULL) { /* error */ } b = ngx_create_temp_buf(pool, 3); if (b == NULL) { /* error */ } b->last = ngx_cpymem(b->last, "foo", 3); cl->buf = b; cl->next = NULL; *ll = cl; return out; }
The connection type ngx_connection_t is a wrapper around a socket descriptor. It includes the following fields:
- fd — Socket descriptor
- data — Arbitrary connection context. Normally, it is a pointer to a higher-level object built on top of the connection, such as an HTTP request or a Stream session.
- read , write — Read and write events for the connection.
- recv , send , recv_chain , send_chain — I/O operations for the connection.
- pool — Connection pool.
- log — Connection log.
- sockaddr , socklen , addr_text — Remote socket address in binary and text forms.
- local_sockaddr , local_socklen — Local socket address in binary form. Initially, these fields are empty. Use the ngx_connection_local_sockaddr() function to get the local socket address.
- proxy_protocol_addr , proxy_protocol_port - PROXY protocol client address and port, if the PROXY protocol is enabled for the connection.
- ssl — SSL context for the connection.
- reusable — Flag indicating the connection is in a state that makes it eligible for reuse.
- close — Flag indicating that the connection is being reused and needs to be closed.
An nginx connection can transparently encapsulate the SSL layer. In this case the connection's ssl field holds a pointer to an ngx_ssl_connection_t structure, keeping all SSL-related data for the connection, including SSL_CTX and SSL . The recv , send , recv_chain , and send_chain handlers are set to SSL-enabled functions as well.
The worker_connections directive in the nginx configuration limits the number of connections per nginx worker. All connection structures are precreated when a worker starts and stored in the connections field of the cycle object. To retrieve a connection structure, use the ngx_get_connection(s, log) function. It takes as its s argument a socket descriptor, which needs to be wrapped in a connection structure.
Because the number of connections per worker is limited, nginx provides a way to grab connections that are currently in use. To enable or disable reuse of a connection, call the ngx_reusable_connection(c, reusable) function. Calling ngx_reusable_connection(c, 1) sets the reuse flag in the connection structure and inserts the connection into the reusable_connections_queue of the cycle. Whenever ngx_get_connection() finds out there are no available connections in the cycle's free_connections list, it calls ngx_drain_connections() to release a specific number of reusable connections. For each such connection, the close flag is set and its read handler is called which is supposed to free the connection by calling ngx_close_connection(c) and make it available for reuse. To exit the state when a connection can be reused ngx_reusable_connection(c, 0) is called. HTTP client connections are an example of reusable connections in nginx; they are marked as reusable until the first request byte is received from the client.
Event object ngx_event_t in nginx provides a mechanism for notification that a specific event has occurred.
Fields in ngx_event_t include:
- data — Arbitrary event context used in event handlers, usually as pointer to a connection related to the event.
- handler — Callback function to be invoked when the event happens.
- write — Flag indicating a write event. Absence of the flag indicates a read event.
- active — Flag indicating that the event is registered for receiving I/O notifications, normally from notification mechanisms like epoll , kqueue , poll .
- ready — Flag indicating that the event has received an I/O notification.
- delayed — Flag indicating that I/O is delayed due to rate limiting.
- timer — Red-black tree node for inserting the event into the timer tree.
- timer_set — Flag indicating that the event timer is set and not yet expired.
- timedout — Flag indicating that the event timer has expired.
- eof — Flag indicating that EOF occurred while reading data.
- pending_eof — Flag indicating that EOF is pending on the socket, even though there may be some data available before it. The flag is delivered via the EPOLLRDHUP epoll event or EV_EOF kqueue flag.
- error — Flag indicating that an error occurred during reading (for a read event) or writing (for a write event).
- cancelable — Timer event flag indicating that the event should be ignored while shutting down the worker. Graceful worker shutdown is delayed until there are no non-cancelable timer events scheduled.
- posted — Flag indicating that the event is posted to a queue.
- queue — Queue node for posting the event to a queue.
Each connection obtained by calling the ngx_get_connection() function has two attached events, c->read and c->write , which are used for receiving notification that the socket is ready for reading or writing. All such events operate in Edge-Triggered mode, meaning that they only trigger notifications when the state of the socket changes. For example, doing a partial read on a socket does not make nginx deliver a repeated read notification until more data arrives on the socket. Even when the underlying I/O notification mechanism is essentially Level-Triggered ( poll , select etc), nginx converts the notifications to Edge-Triggered. To make nginx event notifications consistent across all notifications systems on different platforms, the functions ngx_handle_read_event(rev, flags) and ngx_handle_write_event(wev, lowat) must be called after handling an I/O socket notification or calling any I/O functions on that socket. Normally, the functions are called once at the end of each read or write event handler.
Timer events
An event can be set to send a notification when a timeout expires. The timer used by events counts milliseconds since some unspecified point in the past truncated to ngx_msec_t type. Its current value can be obtained from the ngx_current_msec variable.
The function ngx_add_timer(ev, timer) sets a timeout for an event, ngx_del_timer(ev) deletes a previously set timeout. The global timeout red-black tree ngx_event_timer_rbtree stores all timeouts currently set. The key in the tree is of type ngx_msec_t and is the time when the event occurs. The tree structure enables fast insertion and deletion operations, as well as access to the nearest timeouts, which nginx uses to find out how long to wait for I/O events and for expiring timeout events.
Posted events
An event can be posted which means that its handler will be called at some point later within the current event loop iteration. Posting events is a good practice for simplifying code and escaping stack overflows. Posted events are held in a post queue. The ngx_post_event(ev, q) macro posts the event ev to the post queue q . The ngx_delete_posted_event(ev) macro deletes the event ev from the queue it's currently posted in. Normally, events are posted to the ngx_posted_events queue, which is processed late in the event loop — after all I/O and timer events are already handled. The function ngx_event_process_posted() is called to process an event queue. It calls event handlers until the queue is not empty. This means that a posted event handler can post more events to be processed within the current event loop iteration.
void ngx_my_connection_read(ngx_connection_t *c) { ngx_event_t *rev; rev = c->read; ngx_add_timer(rev, 1000); rev->handler = ngx_my_read_handler; ngx_my_read(rev); } void ngx_my_read_handler(ngx_event_t *rev) { ssize_t n; ngx_connection_t *c; u_char buf[256]; if (rev->timedout) { /* timeout expired */ } c = rev->data; while (rev->ready) { n = c->recv(c, buf, sizeof(buf)); if (n == NGX_AGAIN) { break; } if (n == NGX_ERROR) { /* error */ } /* process buf */ } if (ngx_handle_read_event(rev, 0) != NGX_OK) { /* error */ } }
Except for the nginx master process, all nginx processes do I/O and so have an event loop. (The nginx master process instead spends most of its time in the sigsuspend() call waiting for signals to arrive.) The nginx event loop is implemented in the ngx_process_events_and_timers() function, which is called repeatedly until the process exits.
The event loop has the following stages:
- Find the timeout that is closest to expiring, by calling ngx_event_find_timer() . This function finds the leftmost node in the timer tree and returns the number of milliseconds until the node expires.
- Process I/O events by calling a handler, specific to the event notification mechanism, chosen by nginx configuration. This handler waits for at least one I/O event to happen, but only until the next timeout expires. When a read or write event occurs, the ready flag is set and the event's handler is called. For Linux, the ngx_epoll_process_events() handler is normally used, which calls epoll_wait() to wait for I/O events.
- Expire timers by calling ngx_event_expire_timers() . The timer tree is iterated from the leftmost element to the right until an unexpired timeout is found. For each expired node the timedout event flag is set, the timer_set flag is reset, and the event handler is called
- Process posted events by calling ngx_event_process_posted() . The function repeatedly removes the first element from the posted events queue and calls the element's handler, until the queue is empty.
All nginx processes handle signals as well. Signal handlers only set global variables which are checked after the ngx_process_events_and_timers() call.
There are several types of processes in nginx. The type of a process is kept in the ngx_process global variable, and is one of the following:
NGX_PROCESS_MASTER — The master process, which reads the NGINX configuration, creates cycles, and starts and controls child processes. It does not perform any I/O and responds only to signals. Its cycle function is ngx_master_process_cycle() .
NGX_PROCESS_WORKER — The worker process, which handles client connections. It is started by the master process and responds to its signals and channel commands as well. Its cycle function is ngx_worker_process_cycle() . There can be multiple worker processes, as configured by the worker_processes directive.
NGX_PROCESS_SINGLE — The single process, which exists only in master_process off mode, and is the only process running in that mode. It creates cycles (like the master process does) and handles client connections (like the worker process does). Its cycle function is ngx_single_process_cycle() .
NGX_PROCESS_HELPER — The helper process, of which currently there are two types: cache manager and cache loader. The cycle function for both is ngx_cache_manager_process_cycle() .
The nginx processes handle the following signals:
NGX_SHUTDOWN_SIGNAL ( SIGQUIT on most systems) — Gracefully shutdown. Upon receiving this signal, the master process sends a shutdown signal to all child processes. When no child processes are left, the master destroys the cycle pool and exits. When a worker process receives this signal, it closes all listening sockets and waits until there are no non-cancelable events scheduled, then destroys the cycle pool and exits. When the cache manager or the cache loader process receives this signal, it exits immediately. The ngx_quit variable is set to 1 when a process receives this signal, and is immediately reset after being processed. The ngx_exiting variable is set to 1 while a worker process is in the shutdown state.
NGX_TERMINATE_SIGNAL ( SIGTERM on most systems) — Terminate. Upon receiving this signal, the master process sends a terminate signal to all child processes. If a child process does not exit within 1 second, the master process sends the SIGKILL signal to kill it. When no child processes are left, the master process destroys the cycle pool and exits. When a worker process, the cache manager process or the cache loader process receives this signal, it destroys the cycle pool and exits. The variable ngx_terminate is set to 1 when this signal is received.
NGX_NOACCEPT_SIGNAL ( SIGWINCH on most systems) - Shut down all worker and helper processes. Upon receiving this signal, the master process shuts down its child processes. If a previously started new nginx binary exits, the child processes of the old master are started again. When a worker process receives this signal, it shuts down in debug mode set by the debug_points directive.
NGX_RECONFIGURE_SIGNAL ( SIGHUP on most systems) - Reconfigure. Upon receiving this signal, the master process re-reads the configuration and creates a new cycle based on it. If the new cycle is created successfully, the old cycle is deleted and new child processes are started. Meanwhile, the old child processes receive the NGX_SHUTDOWN_SIGNAL signal. In single-process mode, nginx creates a new cycle, but keeps the old one until there are no longer clients with active connections tied to it. The worker and helper processes ignore this signal.
NGX_REOPEN_SIGNAL ( SIGUSR1 on most systems) — Reopen files. The master process sends this signal to workers, which reopen all open_files related to the cycle.
NGX_CHANGEBIN_SIGNAL ( SIGUSR2 on most systems) — Change the nginx binary. The master process starts a new nginx binary and passes in a list of all listen sockets. The text-format list, passed in the “NGINX” environment variable, consists of descriptor numbers separated with semicolons. The new nginx binary reads the “NGINX” variable and adds the sockets to its init cycle. Other processes ignore this signal.
While all nginx worker processes are able to receive and properly handle POSIX signals, the master process does not use the standard kill() syscall to pass signals to workers and helpers. Instead, nginx uses inter-process socket pairs which allow sending messages between all nginx processes. Currently, however, messages are only sent from the master to its children. The messages carry the standard signals.
It is possible to offload into a separate thread tasks that would otherwise block the nginx worker process. For example, nginx can be configured to use threads to perform file I/O . Another use case is a library that doesn't have asynchronous interface and thus cannot be normally used with nginx. Keep in mind that the threads interface is a helper for the existing asynchronous approach to processing client connections, and by no means intended as a replacement.
To deal with synchronization, the following wrappers over pthreads primitives are available:
- ngx_int_t ngx_thread_mutex_create(ngx_thread_mutex_t *mtx, ngx_log_t *log);
- ngx_int_t ngx_thread_mutex_destroy(ngx_thread_mutex_t *mtx, ngx_log_t *log);
- ngx_int_t ngx_thread_mutex_lock(ngx_thread_mutex_t *mtx, ngx_log_t *log);
- ngx_int_t ngx_thread_mutex_unlock(ngx_thread_mutex_t *mtx, ngx_log_t *log);
- ngx_int_t ngx_thread_cond_create(ngx_thread_cond_t *cond, ngx_log_t *log);
- ngx_int_t ngx_thread_cond_destroy(ngx_thread_cond_t *cond, ngx_log_t *log);
- ngx_int_t ngx_thread_cond_signal(ngx_thread_cond_t *cond, ngx_log_t *log);
- ngx_int_t ngx_thread_cond_wait(ngx_thread_cond_t *cond, ngx_thread_mutex_t *mtx, ngx_log_t *log);
Instead of creating a new thread for each task, nginx implements a thread_pool strategy. Multiple thread pools may be configured for different purposes (for example, performing I/O on different sets of disks). Each thread pool is created at startup and contains a limited number of threads that process a queue of tasks. When a task is completed, a predefined completion handler is called.
The src/core/ngx_thread_pool.h header file contains relevant definitions:
struct ngx_thread_task_s { ngx_thread_task_t *next; ngx_uint_t id; void *ctx; void (*handler)(void *data, ngx_log_t *log); ngx_event_t event; }; typedef struct ngx_thread_pool_s ngx_thread_pool_t; ngx_thread_pool_t *ngx_thread_pool_add(ngx_conf_t *cf, ngx_str_t *name); ngx_thread_pool_t *ngx_thread_pool_get(ngx_cycle_t *cycle, ngx_str_t *name); ngx_thread_task_t *ngx_thread_task_alloc(ngx_pool_t *pool, size_t size); ngx_int_t ngx_thread_task_post(ngx_thread_pool_t *tp, ngx_thread_task_t *task);
At configuration time, a module willing to use threads has to obtain a reference to a thread pool by calling ngx_thread_pool_add(cf, name) , which either creates a new thread pool with the given name or returns a reference to the pool with that name if it already exists.
To add a task into a queue of a specified thread pool tp at runtime, use the ngx_thread_task_post(tp, task) function. To execute a function in a thread, pass parameters and setup a completion handler using the ngx_thread_task_t structure:
typedef struct { int foo; } my_thread_ctx_t; static void my_thread_func(void *data, ngx_log_t *log) { my_thread_ctx_t *ctx = data; /* this function is executed in a separate thread */ } static void my_thread_completion(ngx_event_t *ev) { my_thread_ctx_t *ctx = ev->data; /* executed in nginx event loop */ } ngx_int_t my_task_offload(my_conf_t *conf) { my_thread_ctx_t *ctx; ngx_thread_task_t *task; task = ngx_thread_task_alloc(conf->pool, sizeof(my_thread_ctx_t)); if (task == NULL) { return NGX_ERROR; } ctx = task->ctx; ctx->foo = 42; task->handler = my_thread_func; task->event.handler = my_thread_completion; task->event.data = ctx; if (ngx_thread_task_post(conf->thread_pool, task) != NGX_OK) { return NGX_ERROR; } return NGX_OK; }
Adding new modules
Each standalone nginx module resides in a separate directory that contains at least two files: config and a file with the module source code. The config file contains all information needed for nginx to integrate the module, for example:
ngx_module_type=CORE ngx_module_name=ngx_foo_module ngx_module_srcs="$ngx_addon_dir/ngx_foo_module.c" . auto/module ngx_addon_name=$ngx_module_name
The config file is a POSIX shell script that can set and access the following variables:
- ngx_module_type — Type of module to build. Possible values are CORE , HTTP , HTTP_FILTER , HTTP_INIT_FILTER , HTTP_AUX_FILTER , MAIL , STREAM , or MISC .
- ngx_module_name — Module names. To build multiple modules from a set of source files, specify a whitespace-separated list of names. The first name indicates the name of the output binary for the dynamic module. The names in the list must match the names used in the source code.
- ngx_addon_name — Name of the module as it appears in output on the console from the configure script.
- ngx_module_srcs — Whitespace-separated list of source files used to compile the module. The $ngx_addon_dir variable can be used to represent the path to the module directory.
- ngx_module_incs — Include paths required to build the module
- ngx_module_deps — Whitespace-separated list of the module's dependencies. Usually, it is the list of header files.
- ngx_module_libs — Whitespace-separated list of libraries to link with the module. For example, use ngx_module_libs=-lpthread to link libpthread library. The following macros can be used to link against the same libraries as nginx: LIBXSLT , LIBGD , GEOIP , PCRE , OPENSSL , MD5 , SHA1 , ZLIB , and PERL .
- ngx_module_link — Variable set by the build system to DYNAMIC for a dynamic module or ADDON for a static module and used to determine different actions to perform depending on linking type.
The following modules are typically used as references. The ngx_http_copy_filter_module reads the data for other filter modules and is placed near the bottom of the list so that it is one of the first to be executed. The ngx_http_write_filter_module writes the data to the client socket and is placed near the top of the list, and is the last to be executed.
By default, filter modules are placed before the ngx_http_copy_filter in the module list so that the filter handler is executed after the copy filter handler. For other module types the default is the empty string.
To compile a module into nginx statically, use the --add-module=/path/to/module argument to the configure script. To compile a module for later dynamic loading into nginx, use the --add-dynamic-module=/path/to/module argument.
Core Modules
Modules are the building blocks of nginx, and most of its functionality is implemented as modules. The module source file must contain a global variable of type ngx_module_t , which is defined as follows:
struct ngx_module_s { /* private part is omitted */ void *ctx; ngx_command_t *commands; ngx_uint_t type; ngx_int_t (*init_master)(ngx_log_t *log); ngx_int_t (*init_module)(ngx_cycle_t *cycle); ngx_int_t (*init_process)(ngx_cycle_t *cycle); ngx_int_t (*init_thread)(ngx_cycle_t *cycle); void (*exit_thread)(ngx_cycle_t *cycle); void (*exit_process)(ngx_cycle_t *cycle); void (*exit_master)(ngx_cycle_t *cycle); /* stubs for future extensions are omitted */ };
The omitted private part includes the module version and a signature and is filled using the predefined macro NGX_MODULE_V1 .
Each module keeps its private data in the ctx field, recognizes the configuration directives, specified in the commands array, and can be invoked at certain stages of nginx lifecycle. The module lifecycle consists of the following events:
- Configuration directive handlers are called as they appear in configuration files in the context of the master process.
- After the configuration is parsed successfully, init_module handler is called in the context of the master process. The init_module handler is called in the master process each time a configuration is loaded.
- The master process creates one or more worker processes and the init_process handler is called in each of them.
- When a worker process receives the shutdown or terminate command from the master, it invokes the exit_process handler.
- The master process calls the exit_master handler before exiting.
Because threads are used in nginx only as a supplementary I/O facility with its own API, init_thread and exit_thread handlers are not currently called. There is also no init_master handler, because it would be unnecessary overhead.
The module type defines exactly what is stored in the ctx field. Its value is one of the following types:
- NGX_CORE_MODULE
- NGX_EVENT_MODULE
- NGX_HTTP_MODULE
- NGX_MAIL_MODULE
- NGX_STREAM_MODULE
The NGX_CORE_MODULE is the most basic and thus the most generic and most low-level type of module. The other module types are implemented on top of it and provide a more convenient way to deal with corresponding domains, like handling events or HTTP requests.
The set of core modules includes ngx_core_module , ngx_errlog_module , ngx_regex_module , ngx_thread_pool_module and ngx_openssl_module modules. The HTTP module, the stream module, the mail module and event modules are core modules too. The context of a core module is defined as:
typedef struct { ngx_str_t name; void *(*create_conf)(ngx_cycle_t *cycle); char *(*init_conf)(ngx_cycle_t *cycle, void *conf); } ngx_core_module_t;
where the name is a module name string, create_conf and init_conf are pointers to functions that create and initialize module configuration respectively. For core modules, nginx calls create_conf before parsing a new configuration and init_conf after all configuration is parsed successfully. The typical create_conf function allocates memory for the configuration and sets default values.
For example, a simplistic module called ngx_foo_module might look like this:
/* * Copyright (C) Author. */ #include <ngx_config.h> #include <ngx_core.h> typedef struct { ngx_flag_t enable; } ngx_foo_conf_t; static void *ngx_foo_create_conf(ngx_cycle_t *cycle); static char *ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf); static char *ngx_foo_enable(ngx_conf_t *cf, void *post, void *data); static ngx_conf_post_t ngx_foo_enable_post = { ngx_foo_enable }; static ngx_command_t ngx_foo_commands[] = { { ngx_string("foo_enabled"), NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, 0, offsetof(ngx_foo_conf_t, enable), &ngx_foo_enable_post }, ngx_null_command }; static ngx_core_module_t ngx_foo_module_ctx = { ngx_string("foo"), ngx_foo_create_conf, ngx_foo_init_conf }; ngx_module_t ngx_foo_module = { NGX_MODULE_V1, &ngx_foo_module_ctx, /* module context */ ngx_foo_commands, /* module directives */ NGX_CORE_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static void * ngx_foo_create_conf(ngx_cycle_t *cycle) { ngx_foo_conf_t *fcf; fcf = ngx_pcalloc(cycle->pool, sizeof(ngx_foo_conf_t)); if (fcf == NULL) { return NULL; } fcf->enable = NGX_CONF_UNSET; return fcf; } static char * ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf) { ngx_foo_conf_t *fcf = conf; ngx_conf_init_value(fcf->enable, 0); return NGX_CONF_OK; } static char * ngx_foo_enable(ngx_conf_t *cf, void *post, void *data) { ngx_flag_t *fp = data; if (*fp == 0) { return NGX_CONF_OK; } ngx_log_error(NGX_LOG_NOTICE, cf->log, 0, "Foo Module is enabled"); return NGX_CONF_OK; }
Configuration Directives
The ngx_command_t type defines a single configuration directive. Each module that supports configuration provides an array of such structures that describe how to process arguments and what handlers to call:
typedef struct ngx_command_s ngx_command_t; struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; };
Terminate the array with the special value ngx_null_command . The name is the name of a directive as it appears in the configuration file, for example "worker_processes" or "listen". The type is a bit-field of flags that specify the number of arguments the directive takes, its type, and the context in which it appears. The flags are:
- NGX_CONF_NOARGS — Directive takes no arguments.
- NGX_CONF_1MORE — Directive takes one or more arguments.
- NGX_CONF_2MORE — Directive takes two or more arguments.
- NGX_CONF_TAKE1 .. NGX_CONF_TAKE7 — Directive takes exactly the indicated number of arguments.
- NGX_CONF_TAKE12 , NGX_CONF_TAKE13 , NGX_CONF_TAKE23 , NGX_CONF_TAKE123 , NGX_CONF_TAKE1234 — Directive may take different number of arguments. Options are limited to the given numbers. For example, NGX_CONF_TAKE12 means it takes one or two arguments.
The flags for directive types are:
- NGX_CONF_BLOCK — Directive is a block, that is, it can contain other directives within its opening and closing braces, or even implement its own parser to handle contents inside.
- NGX_CONF_FLAG — Directive takes a boolean value, either on or off .
A directive's context defines where it may appear in the configuration:
- NGX_MAIN_CONF — In the top level context.
- NGX_HTTP_MAIN_CONF — In the http block.
- NGX_HTTP_SRV_CONF — In a server block within the http block.
- NGX_HTTP_LOC_CONF — In a location block within the http block.
- NGX_HTTP_UPS_CONF — In an upstream block within the http block.
- NGX_HTTP_SIF_CONF — In an if block within a server block in the http block.
- NGX_HTTP_LIF_CONF — In an if block within a location block in the http block.
- NGX_HTTP_LMT_CONF — In a limit_except block within the http block.
- NGX_STREAM_MAIN_CONF — In the stream block.
- NGX_STREAM_SRV_CONF — In a server block within the stream block.
- NGX_STREAM_UPS_CONF — In an upstream block within the stream block.
- NGX_MAIL_MAIN_CONF — In the mail block.
- NGX_MAIL_SRV_CONF — In a server block within the mail block.
- NGX_EVENT_CONF — In the event block.
- NGX_DIRECT_CONF — Used by modules that don't create a hierarchy of contexts and only have one global configuration. This configuration is passed to the handler as the conf argument.
The configuration parser uses these flags to throw an error in case of a misplaced directive and calls directive handlers supplied with a proper configuration pointer, so that the same directives in different locations can store their values in distinct places.
The set field defines a handler that processes a directive and stores parsed values into the corresponding configuration. There's a number of functions that perform common conversions:
- ngx_conf_set_flag_slot — Converts the literal strings on and off into an ngx_flag_t value with values 1 or 0, respectively.
- ngx_conf_set_str_slot — Stores a string as a value of the ngx_str_t type.
- ngx_conf_set_str_array_slot — Appends a value to an array ngx_array_t of strings ngx_str_t . The array is created if does not already exist.
- ngx_conf_set_keyval_slot — Appends a key-value pair to an array ngx_array_t of key-value pairs ngx_keyval_t . The first string becomes the key and the second the value. The array is created if it does not already exist.
- ngx_conf_set_num_slot — Converts a directive's argument to an ngx_int_t value.
- ngx_conf_set_size_slot — Converts a size to a size_t value expressed in bytes.
- ngx_conf_set_off_slot — Converts an offset to an off_t value expressed in bytes.
- ngx_conf_set_msec_slot — Converts a time to an ngx_msec_t value expressed in milliseconds.
- ngx_conf_set_sec_slot — Converts a time to a time_t value expressed in in seconds.
- ngx_conf_set_bufs_slot — Converts the two supplied arguments into an ngx_bufs_t object that holds the number and size of buffers.
- ngx_conf_set_enum_slot — Converts the supplied argument into an ngx_uint_t value. The null-terminated array of ngx_conf_enum_t passed in the post field defines the acceptable strings and corresponding integer values.
- ngx_conf_set_bitmask_slot — Converts the supplied arguments into an ngx_uint_t value. The mask values for each argument are ORed producing the result. The null-terminated array of ngx_conf_bitmask_t passed in the post field defines the acceptable strings and corresponding mask values.
- set_path_slot — Converts the supplied arguments to an ngx_path_t value and performs all required initializations. For details, see the documentation for the proxy_temp_path directive.
- set_access_slot — Converts the supplied arguments to a file permissions mask. For details, see the documentation for the proxy_store_access directive.
The conf field defines which configuration structure is passed to the directory handler. Core modules only have the global configuration and set NGX_DIRECT_CONF flag to access it. Modules like HTTP, Stream or Mail create hierarchies of configurations. For example, a module's configuration is created for server , location and if scopes.
- NGX_HTTP_MAIN_CONF_OFFSET — Configuration for the http block.
- NGX_HTTP_SRV_CONF_OFFSET — Configuration for a server block within the http block.
- NGX_HTTP_LOC_CONF_OFFSET — Configuration for a location block within the http .
- NGX_STREAM_MAIN_CONF_OFFSET — Configuration for the stream block.
- NGX_STREAM_SRV_CONF_OFFSET — Configuration for a server block within the stream block.
- NGX_MAIL_MAIN_CONF_OFFSET — Configuration for the mail block.
- NGX_MAIL_SRV_CONF_OFFSET — Configuration for a server block within the mail block.
The offset defines the offset of a field in a module configuration structure that holds values for this particular directive. The typical use is to employ the offsetof() macro.
The post field has two purposes: it may be used to define a handler to be called after the main handler has completed, or to pass additional data to the main handler. In the first case, the ngx_conf_post_t structure needs to be initialized with a pointer to the handler, for example:
static char *ngx_do_foo(ngx_conf_t *cf, void *post, void *data); static ngx_conf_post_t ngx_foo_post = { ngx_do_foo };
The post argument is the ngx_conf_post_t object itself, and the data is a pointer to the value, converted from arguments by the main handler with the appropriate type.
Each HTTP client connection runs through the following stages:
- ngx_event_accept() accepts a client TCP connection. This handler is called in response to a read notification on a listen socket. A new ngx_connection_t object is created at this stage to wrap the newly accepted client socket. Each nginx listener provides a handler to pass the new connection object to. For HTTP connections it's ngx_http_init_connection(c) .
- ngx_http_init_connection() performs early initialization of the HTTP connection. At this stage an ngx_http_connection_t object is created for the connection and its reference is stored in the connection's data field. Later it will be replaced by an HTTP request object. A PROXY protocol parser and the SSL handshake are started at this stage as well.
- ngx_http_wait_request_handler() read event handler is called when data is available on the client socket. At this stage an HTTP request object ngx_http_request_t is created and set to the connection's data field.
- ngx_http_process_request_line() read event handler reads client request line. The handler is set by ngx_http_wait_request_handler() . The data is read into connection's buffer . The size of the buffer is initially set by the directive client_header_buffer_size . The entire client header is supposed to fit in the buffer. If the initial size is not sufficient, a bigger buffer is allocated, with the capacity set by the large_client_header_buffers directive.
- ngx_http_process_request_headers() read event handler, is set after ngx_http_process_request_line() to read the client request header.
- ngx_http_core_run_phases() is called when the request header is completely read and parsed. This function runs request phases from NGX_HTTP_POST_READ_PHASE to NGX_HTTP_CONTENT_PHASE . The last phase is intended to generate a response and pass it along the filter chain. The response is not necessarily sent to the client at this phase. It might remain buffered and be sent at the finalization stage.
- ngx_http_finalize_request() is usually called when the request has generated all the output or produced an error. In the latter case an appropriate error page is looked up and used as the response. If the response is not completely sent to the client by this point, an HTTP writer ngx_http_writer() is activated to finish sending outstanding data.
- ngx_http_finalize_connection() is called when the complete response has been sent to the client and the request can be destroyed. If the client connection keepalive feature is enabled, ngx_http_set_keepalive() is called, which destroys the current request and waits for the next request on the connection. Otherwise, ngx_http_close_request() destroys both the request and the connection.
For each client HTTP request the ngx_http_request_t object is created. Some of the fields of this object are:
connection — Pointer to a ngx_connection_t client connection object. Several requests can reference the same connection object at the same time - one main request and its subrequests. After a request is deleted, a new request can be created on the same connection.
Note that for HTTP connections ngx_connection_t 's data field points back to the request. Such requests are called active, as opposed to the other requests tied to the connection. An active request is used to handle client connection events and is allowed to output its response to the client. Normally, each request becomes active at some point so that it can send its output.
ctx — Array of HTTP module contexts. Each module of type NGX_HTTP_MODULE can store any value (normally, a pointer to a structure) in the request. The value is stored in the ctx array at the module's ctx_index position. The following macros provide a convenient way to get and set request contexts:
- ngx_http_get_module_ctx(r, module) — Returns the module 's context
- ngx_http_set_ctx(r, c, module) — Sets c as the module 's context
- main_conf , srv_conf , loc_conf — Arrays of current request configurations. Configurations are stored at the module's ctx_index positions.
- read_event_handler , write_event_handler - Read and write event handlers for the request. Normally, both the read and write event handlers for an HTTP connection are set to ngx_http_request_handler() . This function calls the read_event_handler and write_event_handler handlers for the currently active request.
- cache — Request cache object for caching the upstream response.
- upstream — Request upstream object for proxying.
- pool — Request pool. The request object itself is allocated in this pool, which is destroyed when the request is deleted. For allocations that need to be available throughout the client connection's lifetime, use ngx_connection_t 's pool instead.
- header_in — Buffer into which the client HTTP request header is read.
- headers_in , headers_out — Input and output HTTP headers objects. Both objects contain the headers field of type ngx_list_t for keeping the raw list of headers. In addition to that, specific headers are available for getting and setting as separate fields, for example content_length_n , status etc.
- request_body — Client request body object.
- start_sec , start_msec — Time point when the request was created, used for tracking request duration.
- method , method_name — Numeric and text representation of the client HTTP request method. Numeric values for methods are defined in src/http/ngx_http_request.h with the macros NGX_HTTP_GET , NGX_HTTP_HEAD , NGX_HTTP_POST , etc.
- http_protocol — Client HTTP protocol version in its original text form (“HTTP/1.0”, “HTTP/1.1” etc).
- http_version — Client HTTP protocol version in numeric form ( NGX_HTTP_VERSION_10 , NGX_HTTP_VERSION_11 , etc.).
- http_major , http_minor — Client HTTP protocol version in numeric form split into major and minor parts.
- request_line , unparsed_uri — Request line and URI in the original client request.
- uri , args , exten — URI, arguments and file extension for the current request. The URI value here might differ from the original URI sent by the client due to normalization. Throughout request processing, these values can change as internal redirects are performed.
- main — Pointer to a main request object. This object is created to process a client HTTP request, as opposed to subrequests, which are created to perform a specific subtask within the main request.
- parent — Pointer to the parent request of a subrequest.
- postponed — List of output buffers and subrequests, in the order in which they are sent and created. The list is used by the postpone filter to provide consistent request output when parts of it are created by subrequests.
- post_subrequest — Pointer to a handler with the context to be called when a subrequest gets finalized. Unused for main requests.
posted_requests — List of requests to be started or resumed, which is done by calling the request's write_event_handler . Normally, this handler holds the request main function, which at first runs request phases and then produces the output.
A request is usually posted by the ngx_http_post_request(r, NULL) call. It is always posted to the main request posted_requests list. The function ngx_http_run_posted_requests(c) runs all requests that are posted in the main request of the passed connection's active request. All event handlers call ngx_http_run_posted_requests , which can lead to new posted requests. Normally, it is called after invoking a request's read or write handler.
- phase_handler — Index of current request phase.
- ncaptures , captures , captures_data — Regex captures produced by the last regex match of the request. A regex match can occur at a number of places during request processing: map lookup, server lookup by SNI or HTTP Host, rewrite, proxy_redirect, etc. Captures produced by a lookup are stored in the above mentioned fields. The field ncaptures holds the number of captures, captures holds captures boundaries and captures_data holds the string against which the regex was matched and which is used to extract captures. After each new regex match, request captures are reset to hold new values.
- count — Request reference counter. The field only makes sense for the main request. Increasing the counter is done by simple r->main->count++ . To decrease the counter, call ngx_http_finalize_request(r, rc) . Creating of a subrequest and running the request body read process both increment the counter.
- subrequests — Current subrequest nesting level. Each subrequest inherits its parent's nesting level, decreased by one. An error is generated if the value reaches zero. The value for the main request is defined by the NGX_HTTP_MAX_SUBREQUESTS constant.
- uri_changes — Number of URI changes remaining for the request. The total number of times a request can change its URI is limited by the NGX_HTTP_MAX_URI_CHANGES constant. With each change the value is decremented until it reaches zero, at which time an error is generated. Rewrites and internal redirects to normal or named locations are considered URI changes.
- blocked — Counter of blocks held on the request. While this value is non-zero, the request cannot be terminated. Currently, this value is increased by pending AIO operations (POSIX AIO and thread operations) and active cache lock.
- buffered — Bitmask showing which modules have buffered the output produced by the request. A number of filters can buffer output; for example, sub_filter can buffer data because of a partial string match, copy filter can buffer data because of the lack of free output buffers etc. As long as this value is non-zero, the request is not finalized pending the flush.
- header_only — Flag indicating that the output does not require a body. For example, this flag is used by HTTP HEAD requests.
keepalive — Flag indicating whether client connection keepalive is supported. The value is inferred from the HTTP version and the value of the “Connection” header.
- header_sent — Flag indicating that the output header has already been sent by the request.
- internal — Flag indicating that the current request is internal. To enter the internal state, a request must pass through an internal redirect or be a subrequest. Internal requests are allowed to enter internal locations.
- allow_ranges — Flag indicating that a partial response can be sent to the client, as requested by the HTTP Range header.
- subrequest_ranges — Flag indicating that a partial response can be sent while a subrequest is being processed.
- single_range — Flag indicating that only a single continuous range of output data can be sent to the client. This flag is usually set when sending a stream of data, for example from a proxied server, and the entire response is not available in one buffer.
- main_filter_need_in_memory , filter_need_in_memory — Flags requesting that the output produced in memory buffers rather than files. This is a signal to the copy filter to read data from file buffers even if sendfile is enabled. The difference between the two flags is the location of the filter modules that set them. Filters called before the postpone filter in the filter chain set filter_need_in_memory , requesting that only the current request output come in memory buffers. Filters called later in the filter chain set main_filter_need_in_memory , requesting that both the main request and all subrequests read files in memory while sending output.
- filter_need_temporary — Flag requesting that the request output be produced in temporary buffers, but not in readonly memory buffers or file buffers. This is used by filters which may change output directly in the buffers where it's sent.
Configuration
Each HTTP module can have three types of configuration:
- Main configuration — Applies to the entire http block. Functions as global settings for a module.
- Server configuration — Applies to a single server block. Functions as server-specific settings for a module.
- Location configuration — Applies to a single location , if or limit_except block. Functions as location-specific settings for a module.
Configuration structures are created at the nginx configuration stage by calling functions, which allocate the structures, initialize them and merge them. The following example shows how to create a simple location configuration for a module. The configuration has one setting, foo , of type unsigned integer.
typedef struct { ngx_uint_t foo; } ngx_http_foo_loc_conf_t; static ngx_http_module_t ngx_http_foo_module_ctx = { NULL, /* preconfiguration */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ ngx_http_foo_create_loc_conf, /* create location configuration */ ngx_http_foo_merge_loc_conf /* merge location configuration */ }; static void * ngx_http_foo_create_loc_conf(ngx_conf_t *cf) { ngx_http_foo_loc_conf_t *conf; conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_foo_loc_conf_t)); if (conf == NULL) { return NULL; } conf->foo = NGX_CONF_UNSET_UINT; return conf; } static char * ngx_http_foo_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) { ngx_http_foo_loc_conf_t *prev = parent; ngx_http_foo_loc_conf_t *conf = child; ngx_conf_merge_uint_value(conf->foo, prev->foo, 1); }
As seen in the example, the ngx_http_foo_create_loc_conf() function creates a new configuration structure, and ngx_http_foo_merge_loc_conf() merges a configuration with configuration from a higher level. In fact, server and location configuration do not exist only at the server and location levels, but are also created for all levels above them. Specifically, a server configuration is also created at the main level and location configurations are created at the main, server, and location levels. These configurations make it possible to specify server- and location-specific settings at any level of an nginx configuration file. Eventually configurations are merged down. A number of macros like NGX_CONF_UNSET and NGX_CONF_UNSET_UINT are provided for indicating a missing setting and ignoring it while merging. Standard nginx merge macros like ngx_conf_merge_value() and ngx_conf_merge_uint_value() provide a convenient way to merge a setting and set the default value if none of the configurations provided an explicit value. For complete list of macros for different types, see src/core/ngx_conf_file.h .
The following macros are available. for accessing configuration for HTTP modules at configuration time. They all take ngx_conf_t reference as the first argument.
- ngx_http_conf_get_module_main_conf(cf, module)
- ngx_http_conf_get_module_srv_conf(cf, module)
- ngx_http_conf_get_module_loc_conf(cf, module)
The following example gets a pointer to a location configuration of standard nginx core module ngx_http_core_module and replaces the location content handler kept in the handler field of the structure.
static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r); static ngx_command_t ngx_http_foo_commands[] = { { ngx_string("foo"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS, ngx_http_foo, 0, 0, NULL }, ngx_null_command }; static char * ngx_http_foo(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_bar_handler; return NGX_CONF_OK; }
The following macros are available for accessing configuration for HTTP modules at runtime.
- ngx_http_get_module_main_conf(r, module)
- ngx_http_get_module_srv_conf(r, module)
- ngx_http_get_module_loc_conf(r, module)
These macros receive a reference to an HTTP request ngx_http_request_t . The main configuration of a request never changes. Server configuration can change from the default after the virtual server for the request is chosen. Location configuration selected for processing a request can change multiple times as a result of a rewrite operation or internal redirect. The following example shows how to access a module's HTTP configuration at runtime.
static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r) { ngx_http_foo_loc_conf_t *flcf; flcf = ngx_http_get_module_loc_conf(r, ngx_http_foo_module); ... }
Each HTTP request passes through a sequence of phases. In each phase a distinct type of processing is performed on the request. Module-specific handlers can be registered in most phases, and many standard nginx modules register their phase handlers as a way to get called at a specific stage of request processing. Phases are processed successively and the phase handlers are called once the request reaches the phase. Following is the list of nginx HTTP phases.
- NGX_HTTP_POST_READ_PHASE — First phase. The ngx_http_realip_module registers its handler at this phase to enable substitution of client addresses before any other module is invoked.
- NGX_HTTP_SERVER_REWRITE_PHASE — Phase where rewrite directives defined in a server block (but outside a location block) are processed. The ngx_http_rewrite_module installs its handler at this phase.
- NGX_HTTP_FIND_CONFIG_PHASE — Special phase where a location is chosen based on the request URI. Before this phase, the default location for the relevant virtual server is assigned to the request, and any module requesting a location configuration receives the configuration for the default server location. This phase assigns a new location to the request. No additional handlers can be registered at this phase.
- NGX_HTTP_REWRITE_PHASE — Same as NGX_HTTP_SERVER_REWRITE_PHASE , but for rewrite rules defined in the location, chosen in the previous phase.
- NGX_HTTP_POST_REWRITE_PHASE — Special phase where the request is redirected to a new location if its URI changed during a rewrite. This is implemented by the request going through the NGX_HTTP_FIND_CONFIG_PHASE again. No additional handlers can be registered at this phase.
- NGX_HTTP_PREACCESS_PHASE — A common phase for different types of handlers, not associated with access control. The standard nginx modules ngx_http_limit_conn_module and ngx_http_limit_req_module register their handlers at this phase.
- NGX_HTTP_ACCESS_PHASE — Phase where it is verified that the client is authorized to make the request. Standard nginx modules such as ngx_http_access_module and ngx_http_auth_basic_module register their handlers at this phase. By default the client must pass the authorization check of all handlers registered at this phase for the request to continue to the next phase. The satisfy directive, can be used to permit processing to continue if any of the phase handlers authorizes the client.
- NGX_HTTP_POST_ACCESS_PHASE — Special phase where the satisfy any directive is processed. If some access phase handlers denied access and none explicitly allowed it, the request is finalized. No additional handlers can be registered at this phase.
- NGX_HTTP_PRECONTENT_PHASE — Phase for handlers to be called prior to generating content. Standard modules such as ngx_http_try_files_module and ngx_http_mirror_module register their handlers at this phase.
- NGX_HTTP_CONTENT_PHASE — Phase where the response is normally generated. Multiple nginx standard modules register their handlers at this phase, including ngx_http_index_module or ngx_http_static_module . They are called sequentially until one of them produces the output. It's also possible to set content handlers on a per-location basis. If the ngx_http_core_module 's location configuration has handler set, it is called as the content handler and the handlers installed at this phase are ignored.
- NGX_HTTP_LOG_PHASE — Phase where request logging is performed. Currently, only the ngx_http_log_module registers its handler at this stage for access logging. Log phase handlers are called at the very end of request processing, right before freeing the request.
Following is the example of a preaccess phase handler.
static ngx_http_module_t ngx_http_foo_module_ctx = { NULL, /* preconfiguration */ ngx_http_foo_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r) { ngx_str_t *ua; ua = r->headers_in->user_agent; if (ua == NULL) { return NGX_DECLINED; } /* reject requests with "User-Agent: foo" */ if (ua->value.len == 3 && ngx_strncmp(ua->value.data, "foo", 3) == 0) { return NGX_HTTP_FORBIDDEN; } return NGX_DECLINED; } static ngx_int_t ngx_http_foo_init(ngx_conf_t *cf) { ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_PREACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_foo_handler; return NGX_OK; }
Phase handlers are expected to return specific codes:
- NGX_OK — Proceed to the next phase.
- NGX_DECLINED — Proceed to the next handler of the current phase. If the current handler is the last in the current phase, move to the next phase.
- NGX_AGAIN , NGX_DONE — Suspend phase handling until some future event which can be an asynchronous I/O operation or just a delay, for example. It is assumed, that phase handling will be resumed later by calling ngx_http_core_run_phases() .
- Any other value returned by the phase handler is treated as a request finalization code, in particular, an HTTP response code. The request is finalized with the code provided.
For some phases, return codes are treated in a slightly different way. At the content phase, any return code other that NGX_DECLINED is considered a finalization code. Any return code from the location content handlers is considered a finalization code. At the access phase, in satisfy any mode, any return code other than NGX_OK , NGX_DECLINED , NGX_AGAIN , NGX_DONE is considered a denial. If no subsequent access handlers allow or deny access with a different code, the denial code will become the finalization code.
Accessing existing variables
Variables can be referenced by index (this is the most common method) or name (see below ). The index is created at configuration stage, when a variable is added to the configuration. To obtain the variable index, use ngx_http_get_variable_index() :
ngx_str_t name; /* ngx_string("foo") */ ngx_int_t index; index = ngx_http_get_variable_index(cf, &name);
Here, cf is a pointer to nginx configuration and name points to a string containing the variable name. The function returns NGX_ERROR on error or a valid index otherwise, which is typically stored somewhere in the module's configuration for future use.
All HTTP variables are evaluated in the context of a given HTTP request, and results are specific to and cached in that HTTP request. All functions that evaluate variables return the ngx_http_variable_value_t type, representing the variable value:
typedef ngx_variable_value_t ngx_http_variable_value_t; typedef struct { unsigned len:28; unsigned valid:1; unsigned no_cacheable:1; unsigned not_found:1; unsigned escape:1; u_char *data; } ngx_variable_value_t;
- len — The length of the value
- data — The value itself
- valid — The value is valid
- not_found — The variable was not found and thus the data and len fields are irrelevant; this can happen, for example, with variables like $arg_foo when a corresponding argument was not passed in a request
- no_cacheable — Do not cache result
- escape — Used internally by the logging module to mark values that require escaping on output.
The ngx_http_get_flushed_variable() and ngx_http_get_indexed_variable() functions are used to obtain the value of a variable. They have the same interface - accepting an HTTP request r as a context for evaluating the variable and an index that identifies it. An example of typical usage:
ngx_http_variable_value_t *v; v = ngx_http_get_flushed_variable(r, index); if (v == NULL || v->not_found) { /* we failed to get value or there is no such variable, handle it */ return NGX_ERROR; } /* some meaningful value is found */
The difference between functions is that the ngx_http_get_indexed_variable() returns a cached value and ngx_http_get_flushed_variable() flushes the cache for non-cacheable variables.
Some modules, such as SSI and Perl, need to deal with variables for which the name is not known at configuration time. An index therefore cannot be used to access them, but the ngx_http_get_variable(r, name, key) function is available. It searches for a variable with a given name and its hash key derived from the name.
Creating variables
To create a variable, use the ngx_http_add_variable() function. It takes as arguments a configuration (where the variable is registered), the variable name and flags that control the function's behaviour:
- NGX_HTTP_VAR_CHANGEABLE — Enables redefinition of the variable: there is no conflict if another module defines a variable with the same name. This allows the set directive to override variables.
- NGX_HTTP_VAR_NOCACHEABLE — Disables caching, which is useful for variables such as $time_local .
- NGX_HTTP_VAR_NOHASH — Indicates that this variable is only accessible by index, not by name. This is a small optimization for use when it is known that the variable is not needed in modules like SSI or Perl.
- NGX_HTTP_VAR_PREFIX — The name of the variable is a prefix. In this case, a handler must implement additional logic to obtain the value of a specific variable. For example, all “ arg_ ” variables are processed by the same handler, which performs lookup in request arguments and returns the value of a specific argument.
The function returns NULL in case of error or a pointer to ngx_http_variable_t otherwise:
struct ngx_http_variable_s { ngx_str_t name; ngx_http_set_variable_pt set_handler; ngx_http_get_variable_pt get_handler; uintptr_t data; ngx_uint_t flags; ngx_uint_t index; };
The get and set handlers are called to obtain or set the variable value, data is passed to variable handlers, and index holds assigned variable index used to reference the variable.
Usually, a null-terminated static array of ngx_http_variable_t structures is created by a module and processed at the preconfiguration stage to add variables into the configuration, for example:
static ngx_http_variable_t ngx_http_foo_vars[] = { { ngx_string("foo_v1"), NULL, ngx_http_foo_v1_variable, 0, 0, 0 }, ngx_http_null_variable }; static ngx_int_t ngx_http_foo_add_variables(ngx_conf_t *cf) { ngx_http_variable_t *var, *v; for (v = ngx_http_foo_vars; v->name.len; v++) { var = ngx_http_add_variable(cf, &v->name, v->flags); if (var == NULL) { return NGX_ERROR; } var->get_handler = v->get_handler; var->data = v->data; } return NGX_OK; }
This function in the example is used to initialize the preconfiguration field of the HTTP module context and is called before the parsing of HTTP configuration, so that the parser can refer to these variables.
The get handler is responsible for evaluating a variable in the context of a specific request, for example:
static ngx_int_t ngx_http_variable_connection(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { u_char *p; p = ngx_pnalloc(r->pool, NGX_ATOMIC_T_LEN); if (p == NULL) { return NGX_ERROR; } v->len = ngx_sprintf(p, "%uA", r->connection->number) - p; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; v->data = p; return NGX_OK; }
It returns NGX_ERROR in case of internal error (for example, failed memory allocation) or NGX_OK otherwise. To learn the status of variable evaluation, inspect the flags in ngx_http_variable_value_t (see the description above ).
The set handler allows setting the property referenced by the variable. For example, the set handler of the $limit_rate variable modifies the request's limit_rate field:
... { ngx_string("limit_rate"), ngx_http_variable_request_set_size, ngx_http_variable_request_get_size, offsetof(ngx_http_request_t, limit_rate), NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOCACHEABLE, 0 }, ... static void ngx_http_variable_request_set_size(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { ssize_t s, *sp; ngx_str_t val; val.len = v->len; val.data = v->data; s = ngx_parse_size(&val); if (s == NGX_ERROR) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "invalid size \"%V\"", &val); return; } sp = (ssize_t *) ((char *) r + data); *sp = s; return; }
Complex values
A complex value, despite its name, provides an easy way to evaluate expressions which can contain text, variables, and their combination.
The complex value description in ngx_http_compile_complex_value is compiled at the configuration stage into ngx_http_complex_value_t which is used at runtime to obtain results of expression evaluation.
ngx_str_t *value; ngx_http_complex_value_t cv; ngx_http_compile_complex_value_t ccv; value = cf->args->elts; /* directive arguments */ ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); ccv.cf = cf; ccv.value = &value[1]; ccv.complex_value = &cv; ccv.zero = 1; ccv.conf_prefix = 1; if (ngx_http_compile_complex_value(&ccv) != NGX_OK) { return NGX_CONF_ERROR; }
Here, ccv holds all parameters that are required to initialize the complex value cv :
- cf — Configuration pointer
- value — String to be parsed (input)
- complex_value — Compiled value (output)
- zero — Flag that enables zero-terminating value
- conf_prefix — Prefixes the result with the configuration prefix (the directory where nginx is currently looking for configuration)
- root_prefix — Prefixes the result with the root prefix (the normal nginx installation prefix)
The zero flag is useful when results are to be passed to libraries that require zero-terminated strings, and prefixes are handy when dealing with filenames.
Upon successful compilation, cv.lengths contains information about the presence of variables in the expression. The NULL value means that the expression contained static text only, and so can be stored in a simple string rather than as a complex value.
The ngx_http_set_complex_value_slot() is a convenient function used to initialize a complex value completely in the directive declaration itself.
At runtime, a complex value can be calculated using the ngx_http_complex_value() function:
ngx_str_t res; if (ngx_http_complex_value(r, &cv, &res) != NGX_OK) { return NGX_ERROR; }
Given the request r and previously compiled value cv , the function evaluates the expression and writes the result to res .
Request redirection
An HTTP request is always connected to a location via the loc_conf field of the ngx_http_request_t structure. This means that at any point the location configuration of any module can be retrieved from the request by calling ngx_http_get_module_loc_conf(r, module) . Request location can change several times during the request's lifetime. Initially, a default server location of the default server is assigned to a request. If the request switches to a different server (chosen by the HTTP “Host” header or SSL SNI extension), the request switches to the default location of that server as well. The next change of the location takes place at the NGX_HTTP_FIND_CONFIG_PHASE request phase. At this phase a location is chosen by request URI among all non-named locations configured for the server. The ngx_http_rewrite_module can change the request URI at the NGX_HTTP_REWRITE_PHASE request phase as a result of the rewrite directive and send the request back to the NGX_HTTP_FIND_CONFIG_PHASE phase for selection of a new location based on the new URI.
It is also possible to redirect a request to a new location at any point by calling one of ngx_http_internal_redirect(r, uri, args) or ngx_http_named_location(r, name) .
The ngx_http_internal_redirect(r, uri, args) function changes the request URI and returns the request to the NGX_HTTP_SERVER_REWRITE_PHASE phase. The request proceeds with a server default location. Later at NGX_HTTP_FIND_CONFIG_PHASE a new location is chosen based on the new request URI.
The following example performs an internal redirect with the new request arguments.
ngx_int_t ngx_http_foo_redirect(ngx_http_request_t *r) { ngx_str_t uri, args; ngx_str_set(&uri, "/foo"); ngx_str_set(&args, "bar=1"); return ngx_http_internal_redirect(r, &uri, &args); }
The function ngx_http_named_location(r, name) redirects a request to a named location. The name of the location is passed as the argument. The location is looked up among all named locations of the current server, after which the requests switches to the NGX_HTTP_REWRITE_PHASE phase.
The following example performs a redirect to a named location @foo.
ngx_int_t ngx_http_foo_named_redirect(ngx_http_request_t *r) { ngx_str_t name; ngx_str_set(&name, "foo"); return ngx_http_named_location(r, &name); }
Both functions - ngx_http_internal_redirect(r, uri, args) and ngx_http_named_location(r, name) can be called when nginx modules have already stored some contexts in a request's ctx field. It's possible for these contexts to become inconsistent with the new location configuration. To prevent inconsistency, all request contexts are erased by both redirect functions.
Calling ngx_http_internal_redirect(r, uri, args) or ngx_http_named_location(r, name) increases the request count . For consistent request reference counting, call ngx_http_finalize_request(r, NGX_DONE) after redirecting the request. This will finalize current request code path and decrease the counter.
Redirected and rewritten requests become internal and can access the internal locations. Internal requests have the internal flag set.
Subrequests
Subrequests are primarily used to insert output of one request into another, possibly mixed with other data. A subrequest looks like a normal request, but shares some data with its parent. In particular, all fields related to client input are shared because a subrequest does not receive any other input from the client. The request field parent for a subrequest contains a link to its parent request and is NULL for the main request. The field main contains a link to the main request in a group of requests.
A subrequest starts in the NGX_HTTP_SERVER_REWRITE_PHASE phase. It passes through the same subsequent phases as a normal request and is assigned a location based on its own URI.
The output header in a subrequest is always ignored. The ngx_http_postpone_filter places the subrequest's output body in the right position relative to other data produced by the parent request.
Subrequests are related to the concept of active requests. A request r is considered active if c->data == r , where c is the client connection object. At any given point, only the active request in a request group is allowed to output its buffers to the client. An inactive request can still send its output to the filter chain, but it does not pass beyond the ngx_http_postpone_filter and remains buffered by that filter until the request becomes active. Here are some rules of request activation:
- Initially, the main request is active.
- The first subrequest of an active request becomes active right after creation.
- The ngx_http_postpone_filter activates the next request in the active request's subrequest list, once all data prior to that request are sent.
- When a request is finalized, its parent is activated.
Create a subrequest by calling the function ngx_http_subrequest(r, uri, args, psr, ps, flags) , where r is the parent request, uri and args are the URI and arguments of the subrequest, psr is the output parameter, which receives the newly created subrequest reference, ps is a callback object for notifying the parent request that the subrequest is being finalized, and flags is bitmask of flags. The following flags are available:
- NGX_HTTP_SUBREQUEST_IN_MEMORY - Output is not sent to the client, but rather stored in memory. The flag only affects subrequests which are processed by one of the proxying modules. After a subrequest is finalized its output is available in r->out of type ngx_buf_t .
- NGX_HTTP_SUBREQUEST_WAITED - The subrequest's done flag is set even if the subrequest is not active when it is finalized. This subrequest flag is used by the SSI filter.
- NGX_HTTP_SUBREQUEST_CLONE - The subrequest is created as a clone of its parent. It is started at the same location and proceeds from the same phase as the parent request.
The following example creates a subrequest with the URI of /foo .
ngx_int_t rc; ngx_str_t uri; ngx_http_request_t *sr; ... ngx_str_set(&uri, "/foo"); rc = ngx_http_subrequest(r, &uri, NULL, &sr, NULL, 0); if (rc == NGX_ERROR) { /* error */ }
This example clones the current request and sets a finalization callback for the subrequest.
ngx_int_t ngx_http_foo_clone(ngx_http_request_t *r) { ngx_http_request_t *sr; ngx_http_post_subrequest_t *ps; ps = ngx_palloc(r->pool, sizeof(ngx_http_post_subrequest_t)); if (ps == NULL) { return NGX_ERROR; } ps->handler = ngx_http_foo_subrequest_done; ps->data = "foo"; return ngx_http_subrequest(r, &r->uri, &r->args, &sr, ps, NGX_HTTP_SUBREQUEST_CLONE); } ngx_int_t ngx_http_foo_subrequest_done(ngx_http_request_t *r, void *data, ngx_int_t rc) { char *msg = (char *) data; ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, "done subrequest r:%p msg:%s rc:%i", r, msg, rc); return rc; }
Subrequests are normally created in a body filter, in which case their output can be treated like the output from any explicit request. This means that eventually the output of a subrequest is sent to the client, after all explicit buffers that are passed before subrequest creation and before any buffers that are passed after creation. This ordering is preserved even for large hierarchies of subrequests. The following example inserts output from a subrequest after all request data buffers, but before the final buffer with the last_buf flag.
ngx_int_t ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_buf_t *b; ngx_uint_t last; ngx_chain_t *cl, out; ngx_http_request_t *sr; ngx_http_foo_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module); if (ctx == NULL) { return ngx_http_next_body_filter(r, in); } last = 0; for (cl = in; cl; cl = cl->next) { if (cl->buf->last_buf) { cl->buf->last_buf = 0; cl->buf->last_in_chain = 1; cl->buf->sync = 1; last = 1; } } /* Output explicit output buffers */ rc = ngx_http_next_body_filter(r, in); if (rc == NGX_ERROR || !last) { return rc; } /* * Create the subrequest. The output of the subrequest * will automatically be sent after all preceding buffers, * but before the last_buf buffer passed later in this function. */ if (ngx_http_subrequest(r, ctx->uri, NULL, &sr, NULL, 0) != NGX_OK) { return NGX_ERROR; } ngx_http_set_ctx(r, NULL, ngx_http_foo_filter_module); /* Output the final buffer with the last_buf flag */ b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } b->last_buf = 1; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); }
A subrequest can also be created for other purposes than data output. For example, the ngx_http_auth_request_module module creates a subrequest at the NGX_HTTP_ACCESS_PHASE phase. To disable output at this point, the header_only flag is set on the subrequest. This prevents the subrequest body from being sent to the client. Note that the subrequest's header is never sent to the client. The result of the subrequest can be analyzed in the callback handler.
Request finalization
An HTTP request is finalized by calling the function ngx_http_finalize_request(r, rc) . It is usually finalized by the content handler after all output buffers are sent to the filter chain. At this point all of the output might not be sent to the client, with some of it remaining buffered somewhere along the filter chain. If it is, the ngx_http_finalize_request(r, rc) function automatically installs a special handler ngx_http_writer(r) to finish sending the output. A request is also finalized in case of an error or if a standard HTTP response code needs to be returned to the client.
The function ngx_http_finalize_request(r, rc) expects the following rc values:
- NGX_DONE - Fast finalization. Decrement the request count and destroy the request if it reaches zero. The client connection can be used for more requests after the current request is destroyed.
- NGX_ERROR , NGX_HTTP_REQUEST_TIME_OUT ( 408 ), NGX_HTTP_CLIENT_CLOSED_REQUEST ( 499 ) - Error finalization. Terminate the request as soon as possible and close the client connection.
- NGX_HTTP_CREATED ( 201 ), NGX_HTTP_NO_CONTENT ( 204 ), codes greater than or equal to NGX_HTTP_SPECIAL_RESPONSE ( 300 ) - Special response finalization. For these values nginx either sends to the client a default response page for the code or performs the internal redirect to an error_page location if that is configured for the code.
- Other codes are considered successful finalization codes and might activate the request writer to finish sending the response body. Once the body is completely sent, the request count is decremented. If it reaches zero, the request is destroyed, but the client connection can still be used for other requests. If count is positive, there are unfinished activities within the request, which will be finalized at a later point.
Request body
For dealing with the body of a client request, nginx provides the ngx_http_read_client_request_body(r, post_handler) and ngx_http_discard_request_body(r) functions. The first function reads the request body and makes it available via the request_body request field. The second function instructs nginx to discard (read and ignore) the request body. One of these functions must be called for every request. Normally, the content handler makes the call.
Reading or discarding the client request body from a subrequest is not allowed. It must always be done in the main request. When a subrequest is created, it inherits the parent's request_body object which can be used by the subrequest if the main request has previously read the request body.
The function ngx_http_read_client_request_body(r, post_handler) starts the process of reading the request body. When the body is completely read, the post_handler callback is called to continue processing the request. If the request body is missing or has already been read, the callback is called immediately. The function ngx_http_read_client_request_body(r, post_handler) allocates the request_body request field of type ngx_http_request_body_t . The field bufs of this object keeps the result as a buffer chain. The body can be saved in memory buffers or file buffers, if the capacity specified by the client_body_buffer_size directive is not enough to fit the entire body in memory.
The following example reads a client request body and returns its size.
ngx_int_t ngx_http_foo_content_handler(ngx_http_request_t *r) { ngx_int_t rc; rc = ngx_http_read_client_request_body(r, ngx_http_foo_init); if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { /* error */ return rc; } return NGX_DONE; } void ngx_http_foo_init(ngx_http_request_t *r) { off_t len; ngx_buf_t *b; ngx_int_t rc; ngx_chain_t *in, out; if (r->request_body == NULL) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } len = 0; for (in = r->request_body->bufs; in; in = in->next) { len += ngx_buf_size(in->buf); } b = ngx_create_temp_buf(r->pool, NGX_OFF_T_LEN); if (b == NULL) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } b->last = ngx_sprintf(b->pos, "%O", len); b->last_buf = (r == r->main) ? 1: 0; b->last_in_chain = 1; r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = b->last - b->pos; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { ngx_http_finalize_request(r, rc); return; } out.buf = b; out.next = NULL; rc = ngx_http_output_filter(r, &out); ngx_http_finalize_request(r, rc); }
The following fields of the request determine how the request body is read:
- request_body_in_single_buf - Read the body to a single memory buffer.
- request_body_in_file_only - Always read the body to a file, even if fits in the memory buffer.
- request_body_in_persistent_file - Do not unlink the file immediately after creation. A file with this flag can be moved to another directory.
- request_body_in_clean_file - Unlink the file when the request is finalized. This can be useful when a file was supposed to be moved to another directory but was not moved for some reason.
- request_body_file_group_access - Enable group access to the file by replacing the default 0600 access mask with 0660.
- request_body_file_log_level - Severity level at which to log file errors.
- request_body_no_buffering - Read the request body without buffering.
The request_body_no_buffering flag enables the unbuffered mode of reading a request body. In this mode, after calling ngx_http_read_client_request_body() , the bufs chain might keep only a part of the body. To read the next part, call the ngx_http_read_unbuffered_request_body(r) function. The return value NGX_AGAIN and the request flag reading_body indicate that more data is available. If bufs is NULL after calling this function, there is nothing to read at the moment. The request callback read_event_handler will be called when the next part of request body is available.
Request body filters
After a request body part is read, it's passed to the request body filter chain by calling the first body filter handler stored in the ngx_http_top_request_body_filter variable. It's assumed that every body handler calls the next handler in the chain until the final handler ngx_http_request_body_save_filter(r, cl) is called. This handler collects the buffers in r->request_body->bufs and writes them to a file if necessary. The last request body buffer has nonzero last_buf flag.
If a filter is planning to delay data buffers, it should set the flag r->request_body->filter_need_buffering to 1 when called for the first time.
Following is an example of a simple request body filter that delays request body by one second.
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> #define NGX_HTTP_DELAY_BODY 1000 typedef struct { ngx_event_t event; ngx_chain_t *out; } ngx_http_delay_body_ctx_t; static ngx_int_t ngx_http_delay_body_filter(ngx_http_request_t *r, ngx_chain_t *in); static void ngx_http_delay_body_cleanup(void *data); static void ngx_http_delay_body_event_handler(ngx_event_t *ev); static ngx_int_t ngx_http_delay_body_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_delay_body_module_ctx = { NULL, /* preconfiguration */ ngx_http_delay_body_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_delay_body_filter_module = { NGX_MODULE_V1, &ngx_http_delay_body_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_request_body_filter_pt ngx_http_next_request_body_filter; static ngx_int_t ngx_http_delay_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_chain_t *cl, *ln; ngx_http_cleanup_t *cln; ngx_http_delay_body_ctx_t *ctx; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "delay request body filter"); ctx = ngx_http_get_module_ctx(r, ngx_http_delay_body_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_delay_body_ctx_t)); if (ctx == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_delay_body_filter_module); r->request_body->filter_need_buffering = 1; } if (ngx_chain_add_copy(r->pool, &ctx->out, in) != NGX_OK) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } if (!ctx->event.timedout) { if (!ctx->event.timer_set) { /* cleanup to remove the timer in case of abnormal termination */ cln = ngx_http_cleanup_add(r, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } cln->handler = ngx_http_delay_body_cleanup; cln->data = ctx; /* add timer */ ctx->event.handler = ngx_http_delay_body_event_handler; ctx->event.data = r; ctx->event.log = r->connection->log; ngx_add_timer(&ctx->event, NGX_HTTP_DELAY_BODY); } return ngx_http_next_request_body_filter(r, NULL); } rc = ngx_http_next_request_body_filter(r, ctx->out); for (cl = ctx->out; cl; /* void */) { ln = cl; cl = cl->next; ngx_free_chain(r->pool, ln); } ctx->out = NULL; return rc; } static void ngx_http_delay_body_cleanup(void *data) { ngx_http_delay_body_ctx_t *ctx = data; if (ctx->event.timer_set) { ngx_del_timer(&ctx->event); } } static void ngx_http_delay_body_event_handler(ngx_event_t *ev) { ngx_connection_t *c; ngx_http_request_t *r; r = ev->data; c = r->connection; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "delay request body event"); ngx_post_event(c->read, &ngx_posted_events); } static ngx_int_t ngx_http_delay_body_init(ngx_conf_t *cf) { ngx_http_next_request_body_filter = ngx_http_top_request_body_filter; ngx_http_top_request_body_filter = ngx_http_delay_body_filter; return NGX_OK; }
In nginx an HTTP response is produced by sending the response header followed by the optional response body. Both header and body are passed through a chain of filters and eventually get written to the client socket. An nginx module can install its handler into the header or body filter chain and process the output coming from the previous handler.
Response header
The ngx_http_send_header(r) function sends the output header. Do not call this function until r->headers_out contains all of the data required to produce the HTTP response header. The status field in r->headers_out must always be set. If the response status indicates that a response body follows the header, content_length_n can be set as well. The default value for this field is -1 , which means that the body size is unknown. In this case, chunked transfer encoding is used. To output an arbitrary header, append the headers list.
static ngx_int_t ngx_http_foo_content_handler(ngx_http_request_t *r) { ngx_int_t rc; ngx_table_elt_t *h; /* send header */ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = 3; /* X-Foo: foo */ h = ngx_list_push(&r->headers_out.headers); if (h == NULL) { return NGX_ERROR; } h->hash = 1; ngx_str_set(&h->key, "X-Foo"); ngx_str_set(&h->value, "foo"); rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; } /* send body */ ... }
Header filters
The ngx_http_send_header(r) function invokes the header filter chain by calling the first header filter handler stored in the ngx_http_top_header_filter variable. It's assumed that every header handler calls the next handler in the chain until the final handler ngx_http_header_filter(r) is called. The final header handler constructs the HTTP response based on r->headers_out and passes it to the ngx_http_writer_filter for output.
To add a handler to the header filter chain, store its address in the global variable ngx_http_top_header_filter at configuration time. The previous handler address is normally stored in a static variable in a module and is called by the newly added handler before exiting.
The following example of a header filter module adds the HTTP header " X-Foo: foo " to every response with status 200 .
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r); static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_foo_header_filter_module_ctx = { NULL, /* preconfiguration */ ngx_http_foo_header_filter_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_foo_header_filter_module = { NGX_MODULE_V1, &ngx_http_foo_header_filter_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_output_header_filter_pt ngx_http_next_header_filter; static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r) { ngx_table_elt_t *h; /* * The filter handler adds "X-Foo: foo" header * to every HTTP 200 response */ if (r->headers_out.status != NGX_HTTP_OK) { return ngx_http_next_header_filter(r); } h = ngx_list_push(&r->headers_out.headers); if (h == NULL) { return NGX_ERROR; } h->hash = 1; ngx_str_set(&h->key, "X-Foo"); ngx_str_set(&h->value, "foo"); return ngx_http_next_header_filter(r); } static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf) { ngx_http_next_header_filter = ngx_http_top_header_filter; ngx_http_top_header_filter = ngx_http_foo_header_filter; return NGX_OK; }
Response body
To send the response body, call the ngx_http_output_filter(r, cl) function. The function can be called multiple times. Each time, it sends a part of the response body in the form of a buffer chain. Set the last_buf flag in the last body buffer.
The following example produces a complete HTTP response with "foo" as its body. For the example to work as subrequest as well as a main request, the last_in_chain flag is set in the last buffer of the output. The last_buf flag is set only for the main request because the last buffer for a subrequest does not end the entire output.
static ngx_int_t ngx_http_bar_content_handler(ngx_http_request_t *r) { ngx_int_t rc; ngx_buf_t *b; ngx_chain_t out; /* send header */ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = 3; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; } /* send body */ b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } b->last_buf = (r == r->main) ? 1: 0; b->last_in_chain = 1; b->memory = 1; b->pos = (u_char *) "foo"; b->last = b->pos + 3; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); }
Response body filters
The function ngx_http_output_filter(r, cl) invokes the body filter chain by calling the first body filter handler stored in the ngx_http_top_body_filter variable. It's assumed that every body handler calls the next handler in the chain until the final handler ngx_http_write_filter(r, cl) is called.
A body filter handler receives a chain of buffers. The handler is supposed to process the buffers and pass a possibly new chain to the next handler. It's worth noting that the chain links ngx_chain_t of the incoming chain belong to the caller, and must not be reused or changed. Right after the handler completes, the caller can use its output chain links to keep track of the buffers it has sent. To save the buffer chain or to substitute some buffers before passing to the next filter, a handler needs to allocate its own chain links.
Following is an example of a simple body filter that counts the number of bytes in the body. The result is available as the $counter variable which can be used in the access log.
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> typedef struct { off_t count; } ngx_http_counter_filter_ctx_t; static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in); static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf); static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_counter_filter_module_ctx = { ngx_http_counter_add_variables, /* preconfiguration */ ngx_http_counter_filter_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_counter_filter_module = { NGX_MODULE_V1, &ngx_http_counter_filter_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_output_body_filter_pt ngx_http_next_body_filter; static ngx_str_t ngx_http_counter_name = ngx_string("counter"); static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_chain_t *cl; ngx_http_counter_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_counter_filter_ctx_t)); if (ctx == NULL) { return NGX_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_counter_filter_module); } for (cl = in; cl; cl = cl->next) { ctx->count += ngx_buf_size(cl->buf); } return ngx_http_next_body_filter(r, in); } static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { u_char *p; ngx_http_counter_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module); if (ctx == NULL) { v->not_found = 1; return NGX_OK; } p = ngx_pnalloc(r->pool, NGX_OFF_T_LEN); if (p == NULL) { return NGX_ERROR; } v->data = p; v->len = ngx_sprintf(p, "%O", ctx->count) - p; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; return NGX_OK; } static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf) { ngx_http_variable_t *var; var = ngx_http_add_variable(cf, &ngx_http_counter_name, 0); if (var == NULL) { return NGX_ERROR; } var->get_handler = ngx_http_counter_variable; return NGX_OK; } static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf) { ngx_http_next_body_filter = ngx_http_top_body_filter; ngx_http_top_body_filter = ngx_http_counter_body_filter; return NGX_OK; }
Building filter modules
When writing a body or header filter, pay special attention to the filter's position in the filter order. There's a number of header and body filters registered by nginx standard modules. The nginx standard modules register a number of head and body filters and it's important to register a new filter module in the right place with respect to them. Normally, modules register filters in their postconfiguration handlers. The order in which filters are called during processing is obviously the reverse of the order in which they are registered.
For third-party filter modules nginx provides a special slot HTTP_AUX_FILTER_MODULES . To register a filter module in this slot, set the ngx_module_type variable to HTTP_AUX_FILTER in the module's configuration.
The following example shows a filter module config file assuming for a module with just one source file, ngx_http_foo_filter_module.c .
ngx_module_type=HTTP_AUX_FILTER ngx_module_name=ngx_http_foo_filter_module ngx_module_srcs="$ngx_addon_dir/ngx_http_foo_filter_module.c" . auto/module
Buffer reuse
When issuing or altering a stream of buffers, it's often desirable to reuse the allocated buffers. A standard and widely adopted approach in nginx code is to keep two buffer chains for this purpose: free and busy . The free chain keeps all free buffers, which can be reused. The busy chain keeps all buffers sent by the current module that are still in use by some other filter handler. A buffer is considered in use if its size is greater than zero. Normally, when a buffer is consumed by a filter, its pos (or file_pos for a file buffer) is moved towards last ( file_last for a file buffer). Once a buffer is completely consumed, it's ready to be reused. To add newly freed buffers to the free chain it's enough to iterate over the busy chain and move the zero size buffers at the head of it to free . This operation is so common that there is a special function for it, ngx_chain_update_chains(free, busy, out, tag) . The function appends the output chain out to busy and moves free buffers from the top of busy to free . Only the buffers with the specified tag are reused. This lets a module reuse only the buffers that it allocated itself.
The following example is a body filter that inserts the string “foo” before each incoming buffer. The new buffers allocated by the module are reused if possible. Note that for this example to work properly, setting up a header filter and resetting content_length_n to -1 is also required, but the relevant code is not provided here.
typedef struct { ngx_chain_t *free; ngx_chain_t *busy; } ngx_http_foo_filter_ctx_t; ngx_int_t ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_buf_t *b; ngx_chain_t *cl, *tl, *out, **ll; ngx_http_foo_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_foo_filter_ctx_t)); if (ctx == NULL) { return NGX_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_foo_filter_module); } /* create a new chain "out" from "in" with all the changes */ ll = &out; for (cl = in; cl; cl = cl->next) { /* append "foo" in a reused buffer if possible */ tl = ngx_chain_get_free_buf(r->pool, &ctx->free); if (tl == NULL) { return NGX_ERROR; } b = tl->buf; b->tag = (ngx_buf_tag_t) &ngx_http_foo_filter_module; b->memory = 1; b->pos = (u_char *) "foo"; b->last = b->pos + 3; *ll = tl; ll = &tl->next; /* append the next incoming buffer */ tl = ngx_alloc_chain_link(r->pool); if (tl == NULL) { return NGX_ERROR; } tl->buf = cl->buf; *ll = tl; ll = &tl->next; } *ll = NULL; /* send the new chain */ rc = ngx_http_next_body_filter(r, out); /* update "busy" and "free" chains for reuse */ ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &out, (ngx_buf_tag_t) &ngx_http_foo_filter_module); return rc; }
Load balancing
The ngx_http_upstream_module provides the basic functionality needed to pass requests to remote servers. Modules that implement specific protocols, such as HTTP or FastCGI, use this functionality. The module also provides an interface for creating custom load-balancing modules and implements a default round-robin method.
The least_conn and hash modules implement alternative load-balancing methods, but are actually implemented as extensions of the upstream round-robin module and share a lot of code with it, such as the representation of a server group. The keepalive module is an independent module that extends upstream functionality.
The ngx_http_upstream_module can be configured explicitly by placing the corresponding upstream block into the configuration file, or implicitly by using directives such as proxy_pass that accept a URL that gets evaluated at some point into a list of servers. The alternative load-balancing methods are available only with an explicit upstream configuration. The upstream module configuration has its own directive context NGX_HTTP_UPS_CONF . The structure is defined as follows:
struct ngx_http_upstream_srv_conf_s { ngx_http_upstream_peer_t peer; void **srv_conf; ngx_array_t *servers; /* ngx_http_upstream_server_t */ ngx_uint_t flags; ngx_str_t host; u_char *file_name; ngx_uint_t line; in_port_t port; ngx_uint_t no_port; /* unsigned no_port:1 */ #if (NGX_HTTP_UPSTREAM_ZONE) ngx_shm_zone_t *shm_zone; #endif };
- srv_conf — Configuration context of upstream modules.
- servers — Array of ngx_http_upstream_server_t , the result of parsing a set of server directives in the upstream block.
- NGX_HTTP_UPSTREAM_CREATE — Distinguishes explicitly defined upstreams from those that are automatically created by the proxy_pass directive and “friends” (FastCGI, SCGI, etc.)
- NGX_HTTP_UPSTREAM_WEIGHT — The “ weight ” parameter is supported
- NGX_HTTP_UPSTREAM_MAX_FAILS — The “ max_fails ” parameter is supported
- NGX_HTTP_UPSTREAM_FAIL_TIMEOUT — The “ fail_timeout ” parameter is supported
- NGX_HTTP_UPSTREAM_DOWN — The “ down ” parameter is supported
- NGX_HTTP_UPSTREAM_BACKUP — The “ backup ” parameter is supported
- NGX_HTTP_UPSTREAM_MAX_CONNS — The “ max_conns ” parameter is supported
- host — Name of the upstream.
- file_name, line — Name of the configuration file and the line where the upstream block is located.
- port and no_port — Not used for explicitly defined upstream groups.
- shm_zone — Shared memory zone used by this upstream group, if any.
typedef struct { ngx_http_upstream_init_pt init_upstream; ngx_http_upstream_init_peer_pt init; void *data; } ngx_http_upstream_peer_t;
- init_upstream(cf, us) — Configuration-time method responsible for initializing a group of servers and initializing the init() method in case of success. A typical load-balancing module uses a list of servers in the upstream block to create an efficient data structure that it uses and saves its own configuration to the data field.
- init(r, us) — Initializes a per-request ngx_http_upstream_peer_t.peer structure that is used for load balancing (not to be confused with the ngx_http_upstream_srv_conf_t.peer described above which is per-upstream). It is passed as the data argument to all callbacks that deal with server selection.
When nginx has to pass a request to another host for processing, it uses the configured load-balancing method to obtain an address to connect to. The method is obtained from the ngx_http_upstream_t.peer object of type ngx_peer_connection_t :
struct ngx_peer_connection_s { ... struct sockaddr *sockaddr; socklen_t socklen; ngx_str_t *name; ngx_uint_t tries; ngx_event_get_peer_pt get; ngx_event_free_peer_pt free; ngx_event_notify_peer_pt notify; void *data; #if (NGX_SSL || NGX_COMPAT) ngx_event_set_peer_session_pt set_session; ngx_event_save_peer_session_pt save_session; #endif ... };
The structure has the following fields:
- sockaddr , socklen , name — Address of the upstream server to connect to; this is the output parameter of a load-balancing method.
- data — The per-request data of a load-balancing method; keeps the state of the selection algorithm and usually includes the link to the upstream configuration. It is passed as an argument to all methods that deal with server selection (see below ).
- tries — Allowed number of attempts to connect to an upstream server.
- get , free , notify , set_session , and save_session - Methods of the load-balancing module, described below.
All methods accept at least two arguments: a peer connection object pc and the data created by ngx_http_upstream_srv_conf_t.peer.init() . Note that it might differ from pc.data due to “chaining” of load-balancing modules.
- NGX_OK — Server was selected.
- NGX_ERROR — Internal error occurred.
- NGX_BUSY — no servers are currently available. This can happen due to many reasons, including: the dynamic server group is empty, all servers in the group are in the failed state, or all servers in the group are already handling the maximum number of connections.
- NGX_DONE — the underlying connection was reused and there is no need to create a new connection to the upstream server. This value is set by the keepalive module.
- NGX_PEER_FAILED — Attempt was unsuccessful
- NGX_PEER_NEXT — A special case when upstream server returns codes 403 or 404 , which are not considered a failure .
- NGX_PEER_KEEPALIVE — Currently unused
- notify(pc, data, type) — Currently unused in the OSS version.
- set_session(pc, data) and save_session(pc, data) — SSL-specific methods that enable caching sessions to upstream servers. The implementation is provided by the round-robin balancing method.
The nginx-dev-examples repository provides nginx module examples.
General rules
- maximum text width is 80 characters
- indentation is 4 spaces
- no tabs, no trailing spaces
- list elements on the same line are separated with spaces
- hexadecimal literals are lowercase
- file names, function and type names, and global variables have the ngx_ or more specific prefix such as ngx_http_ and ngx_mail_
size_t ngx_utf8_length(u_char *p, size_t n) { u_char c, *last; size_t len; last = p + n; for (len = 0; p < last; len++) { c = *p; if (c < 0x80) { p++; continue; } if (ngx_utf8_decode(&p, last - p) > 0x10ffff) { /* invalid UTF-8 */ return n; } } return len; }
A typical source file may contain the following sections separated by two empty lines:
- copyright statements
- preprocessor definitions
- type definitions
- function prototypes
- variable definitions
- function definitions
Copyright statements look like this:
/* * Copyright (C) Author Name * Copyright (C) Organization, Inc. */
If the file is modified significantly, the list of authors should be updated, the new author is added to the top.
The ngx_config.h and ngx_core.h files are always included first, followed by one of ngx_http.h , ngx_stream.h , or ngx_mail.h . Then follow optional external header files:
#include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> #include <libxml/parser.h> #include <libxml/tree.h> #include <libxslt/xslt.h> #if (NGX_HAVE_EXSLT) #include <libexslt/exslt.h> #endif
Header files should include the so called "header protection":
#ifndef _NGX_PROCESS_CYCLE_H_INCLUDED_ #define _NGX_PROCESS_CYCLE_H_INCLUDED_ ... #endif /* _NGX_PROCESS_CYCLE_H_INCLUDED_ */
- “ // ” comments are not used
- text is written in English, American spelling is preferred
/* * The red-black tree code is based on the algorithm described in * the "Introduction to Algorithms" by Cormen, Leiserson and Rivest. */
/* find the server configuration for the address:port */
Preprocessor
Macro names start from ngx_ or NGX_ (or more specific) prefix. Macro names for constants are uppercase. Parameterized macros and macros for initializers are lowercase. The macro name and value are separated by at least two spaces:
#define NGX_CONF_BUFFER 4096 #define ngx_buf_in_memory(b) (b->temporary || b->memory || b->mmap) #define ngx_buf_size(b) \ (ngx_buf_in_memory(b) ? (off_t) (b->last - b->pos): \ (b->file_last - b->file_pos)) #define ngx_null_string { 0, NULL }
Conditions are inside parentheses, negation is outside:
#if (NGX_HAVE_KQUEUE) ... #elif ((NGX_HAVE_DEVPOLL && !(NGX_TEST_BUILD_DEVPOLL)) \ || (NGX_HAVE_EVENTPORT && !(NGX_TEST_BUILD_EVENTPORT))) ... #elif (NGX_HAVE_EPOLL && !(NGX_TEST_BUILD_EPOLL)) ... #elif (NGX_HAVE_POLL) ... #else /* select */ ... #endif /* NGX_HAVE_KQUEUE */
Type names end with the “ _t ” suffix. A defined type name is separated by at least two spaces:
typedef ngx_uint_t ngx_rbtree_key_t;
Structure types are defined using typedef . Inside structures, member types and names are aligned:
Keep alignment identical among different structures in the file. A structure that points to itself has the name, ending with “ _s ”. Adjacent structure definitions are separated with two empty lines:
typedef struct ngx_list_part_s ngx_list_part_t; struct ngx_list_part_s { void *elts; ngx_uint_t nelts; ngx_list_part_t *next; }; typedef struct { ngx_list_part_t *last; ngx_list_part_t part; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_list_t;
Each structure member is declared on its own line:
typedef struct { ngx_uint_t hash; ngx_str_t key; ngx_str_t value; u_char *lowcase_key; } ngx_table_elt_t;
Function pointers inside structures have defined types ending with “ _pt ”:
typedef ssize_t (*ngx_recv_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); typedef ssize_t (*ngx_send_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ngx_chain_t *(*ngx_send_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); typedef struct { ngx_recv_pt recv; ngx_recv_chain_pt recv_chain; ngx_recv_pt udp_recv; ngx_send_pt send; ngx_send_pt udp_send; ngx_send_chain_pt udp_send_chain; ngx_send_chain_pt send_chain; ngx_uint_t flags; } ngx_os_io_t;
Enumerations have types ending with “ _e ”:
typedef enum { ngx_http_fastcgi_st_version = 0, ngx_http_fastcgi_st_type, ... ngx_http_fastcgi_st_padding } ngx_http_fastcgi_state_e;
Variables are declared sorted by length of a base type, then alphabetically. Type names and variable names are aligned. The type and name “columns” are separated with two spaces. Large arrays are put at the end of a declaration block:
u_char | | *rv, *p; ngx_conf_t | | *cf; ngx_uint_t | | i, j, k; unsigned int | | len; struct sockaddr | | *sa; const unsigned char | | *data; ngx_peer_connection_t | | *pc; ngx_http_core_srv_conf_t | |**cscfp; ngx_http_upstream_srv_conf_t| | *us, *uscf; u_char | | text[NGX_SOCKADDR_STRLEN];
Static and global variables may be initialized on declaration:
static ngx_str_t ngx_http_memcached_key = ngx_string("memcached_key");
static ngx_uint_t mday[] = { 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 };
static uint32_t ngx_crc32_table16[] = { 0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac, ... 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c };
There is a bunch of commonly used type/name combinations:
u_char *rv; ngx_int_t rc; ngx_conf_t *cf; ngx_connection_t *c; ngx_http_request_t *r; ngx_peer_connection_t *pc; ngx_http_upstream_srv_conf_t *us, *uscf;
All functions (even static ones) should have prototypes. Prototypes include argument names. Long prototypes are wrapped with a single indentation on continuation lines:
static char *ngx_http_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static ngx_int_t ngx_http_init_phases(ngx_conf_t *cf, ngx_http_core_main_conf_t *cmcf); static char *ngx_http_merge_servers(ngx_conf_t *cf, ngx_http_core_main_conf_t *cmcf, ngx_http_module_t *module, ngx_uint_t ctx_index);
The function name in a definition starts with a new line. The function body opening and closing braces are on separate lines. The body of a function is indented. There are two empty lines between functions:
static ngx_int_t ngx_http_find_virtual_server(ngx_http_request_t *r, u_char *host, size_t len) { ... } static ngx_int_t ngx_http_add_addresses(ngx_conf_t *cf, ngx_http_core_srv_conf_t *cscf, ngx_http_conf_port_t *port, ngx_http_listen_opt_t *lsopt) { ... }
There is no space after the function name and opening parenthesis. Long function calls are wrapped such that continuation lines start from the position of the first function argument. If this is impossible, format the first continuation line such that it ends at position 79:
ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http header: \"%V: %V\"", &h->key, &h->value); hc->busy = ngx_palloc(r->connection->pool, cscf->large_client_header_buffers.num * sizeof(ngx_buf_t *));
The ngx_inline macro should be used instead of inline :
static ngx_inline void ngx_cpuid(uint32_t i, uint32_t *buf);
Expressions
Binary operators except “ . ” and “ −> ” should be separated from their operands by one space. Unary operators and subscripts are not separated from their operands by spaces:
width = width * 10 + (*fmt++ - '0');
ch = (u_char) ((decoded << 4) + (ch - '0'));
r->exten.data = &r->uri.data[i + 1];
Type casts are separated by one space from casted expressions. An asterisk inside type cast is separated with space from type name:
len = ngx_sock_ntop((struct sockaddr *) sin6, p, len, 1);
If an expression does not fit into single line, it is wrapped. The preferred point to break a line is a binary operator. The continuation line is lined up with the start of expression:
if (status == NGX_HTTP_MOVED_PERMANENTLY || status == NGX_HTTP_MOVED_TEMPORARILY || status == NGX_HTTP_SEE_OTHER || status == NGX_HTTP_TEMPORARY_REDIRECT || status == NGX_HTTP_PERMANENT_REDIRECT) { ... }
p->temp_file->warn = "an upstream response is buffered " "to a temporary file";
As a last resort, it is possible to wrap an expression so that the continuation line ends at position 79:
hinit->hash = ngx_pcalloc(hinit->pool, sizeof(ngx_hash_wildcard_t) + size * sizeof(ngx_hash_elt_t *));
The above rules also apply to sub-expressions, where each sub-expression has its own indentation level:
if (((u->conf->cache_use_stale & NGX_HTTP_UPSTREAM_FT_UPDATING) || c->stale_updating) && !r->background && u->conf->cache_background_update) { ... }
Sometimes, it is convenient to wrap an expression after a cast. In this case, the continuation line is indented:
node = (ngx_rbtree_node_t *) ((u_char *) lr - offsetof(ngx_rbtree_node_t, color));
Pointers are explicitly compared to NULL (not 0 ):
if (ptr != NULL) { ... }
Conditionals and Loops
The “ if ” keyword is separated from the condition by one space. Opening brace is located on the same line, or on a dedicated line if the condition takes several lines. Closing brace is located on a dedicated line, optionally followed by “ else if / else ”. Usually, there is an empty line before the “ else if / else ” part:
if (node->left == sentinel) { temp = node->right; subst = node; } else if (node->right == sentinel) { temp = node->left; subst = node; } else { subst = ngx_rbtree_min(node->right, sentinel); if (subst->left != sentinel) { temp = subst->left; } else { temp = subst->right; } }
Similar formatting rules are applied to “ do ” and “ while ” loops:
while (p < last && *p == ' ') { p++; }
do { ctx->node = rn; ctx = ctx->next; } while (ctx);
The “ switch ” keyword is separated from the condition by one space. Opening brace is located on the same line. Closing brace is located on a dedicated line. The “ case ” keywords are lined up with “ switch ”:
switch (ch) { case '!': looked = 2; state = ssi_comment0_state; break; case '<': copy_end = p; break; default: copy_end = p; looked = 0; state = ssi_start_state; break; }
Most “ for ” loops are formatted like this:
for (i = 0; i < ccf->env.nelts; i++) { ... }
for (q = ngx_queue_head(locations); q != ngx_queue_sentinel(locations); q = ngx_queue_next(q)) { ... }
If some part of the “ for ” statement is omitted, this is indicated by the “ /* void */ ” comment:
for (i = 0; /* void */ ; i++) { ... }
A loop with an empty body is also indicated by the “ /* void */ ” comment which may be put on the same line:
for (cl = *busy; cl->next; cl = cl->next) { /* void */ }
An endless loop looks like this:
for ( ;; ) { ... }
Labels are surrounded with empty lines and are indented at the previous level:
if (i == 0) { u->err = "host not found"; goto failed; } u->addrs = ngx_pcalloc(pool, i * sizeof(ngx_addr_t)); if (u->addrs == NULL) { goto failed; } u->naddrs = i; ... return NGX_OK; failed: freeaddrinfo(res); return NGX_ERROR;
Debugging memory issues
To debug memory issues such as buffer overruns or use-after-free errors, you can use the AddressSanitizer (ASan) supported by some modern compilers. To enable ASan with gcc and clang , use the -fsanitize=address compiler and linker option. When building nginx, this can be done by adding the option to --with-cc-opt and --with-ld-opt parameters of the configure script.
Since most allocations in nginx are made from nginx internal pool , enabling ASan may not always be enough to debug memory issues. The internal pool allocates a big chunk of memory from the system and cuts smaller allocations from it. However, this mechanism can be disabled by setting the NGX_DEBUG_PALLOC macro to 1 . In this case, allocations are passed directly to the system allocator giving it full control over the buffers boundaries.
The following configuration line summarizes the information provided above. It is recommended while developing third-party modules and testing nginx on different platforms.
auto/configure --with-cc-opt='-fsanitize=address -DNGX_DEBUG_PALLOC=1' --with-ld-opt=-fsanitize=address
Common Pitfalls
Writing a c module.
The most common pitfall is an attempt to write a full-fledged C module when it can be avoided. In most cases your task can be accomplished by creating a proper configuration. If writing a module is inevitable, try to make it as small and simple as possible. For example, a module can only export some variables .
Before starting a module, consider the following questions:
- Is it possible to implement a desired feature using already available modules ?
- Is it possible to solve an issue using built-in scripting languages, such as Perl or njs ?
The most used string type in nginx, ngx_str_t is not a C-Style zero-terminated string. You cannot pass the data to standard C library functions such as strlen() or strstr() . Instead, nginx counterparts that accept either ngx_str_t should be used or pointer to data and a length. However, there is a case when ngx_str_t holds a pointer to a zero-terminated string: strings that come as a result of configuration file parsing are zero-terminated.
Global Variables
Avoid using global variables in your modules. Most likely this is an error to have a global variable. Any global data should be tied to a configuration cycle and be allocated from the corresponding memory pool . This allows nginx to perform graceful configuration reloads. An attempt to use global variables will likely break this feature, because it will be impossible to have two configurations at the same time and get rid of them. Sometimes global variables are required. In this case, special attention is needed to manage reconfiguration properly. Also, check if libraries used by your code have implicit global state that may be broken on reload.
Manual Memory Management
Instead of dealing with malloc/free approach which is error prone, learn how to use nginx pools . A pool is created and tied to an object - configuration , cycle , connection , or HTTP request . When the object is destroyed, the associated pool is destroyed too. So when working with an object, it is possible to allocate the amount needed from the corresponding pool and don't care about freeing memory even in case of errors.
It is recommended to avoid using threads in nginx because it will definitely break things: most nginx functions are not thread-safe. It is expected that a thread will be executing only system calls and thread-safe library functions. If you need to run some code that is not related to client request processing, the proper way is to schedule a timer in the init_process module handler and perform required actions in timer handler. Internally nginx makes use of threads to boost IO-related operations, but this is a special case with a lot of limitations.
Blocking Libraries
A common mistake is to use libraries that are blocking internally. Most libraries out there are synchronous and blocking by nature. In other words, they perform one operation at a time and waste time waiting for response from other peer. As a result, when a request is processed with such library, whole nginx worker is blocked, thus destroying performance. Use only libraries that provide asynchronous interface and don't block whole process.
HTTP Requests to External Services
Often modules need to perform an HTTP call to some external service. A common mistake is to use some external library, such as libcurl, to perform the HTTP request. It is absolutely unnecessary to bring a huge amount of external (probably blocking !) code for the task which can be accomplished by nginx itself.
There are two basic usage scenarios when an external request is needed:
- in the context of processing a client request (for example, in content handler)
- in the context of a worker process (for example, timer handler)
In the first case, the best is to use subrequests API . Instead of directly accessing external service, you declare a location in nginx configuration and direct your subrequest to this location. This location is not limited to proxying requests, but may contain other nginx directives. An example of such approach is the auth_request directive implemented in ngx_http_auth_request module .
For the second case, it is possible to use basic HTTP client functionality available in nginx. For example, OCSP module implements simple HTTP client.
- No suggested jump to results
- Notifications
A hello world module for learning Nginx module development
perusio/nginx-hello-world-module
Name already in use.
Use Git or checkout with SVN using the web URL.
Work fast with our official CLI. Learn more .
- Open with GitHub Desktop
- Download ZIP
Sign In Required
Please sign in to use Codespaces.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching Xcode
If nothing happens, download Xcode and try again.
Launching Visual Studio Code
Your codespace will open once ready.
There was a problem preparing your codespace, please try again.
Latest commit
Hello world module for nginx, introduction.
This module serves as a learning exercise for me, and hopefully for others too, when doing Nginx module development.
I stole the code and added some notes using mostly Evan Miller's Nginx Module Development Guide . Also helpful is the translation of Vhalery Kholodov's Nginx Module Guide done by Antoine Bonavita that also mantains a Nginx Discovery blog to document his journey on Nginx module development.
Installation
Configure Nginx adding this module with:
Build Nginx as usual with make .
Configure the module. There's only one directive hello_world that is supported in the location context only.
Now doing something like:
should return the hello world string as the response body.
Further reading
English: You can go to the Nginx Planet and get a lot of Nginx related information.
Russian: The Habrahabr Nginx Blog a treasure trove of Nginx related discussions. Use google translate or Babelfish if you don't read Russian.
Add an argument to the hello_world directive specifying the language. For example:
en: hello world
fr: bounjour monde
pt: olá mundo
es: hola mundo
Make the hello_world directive be also defined in the if in location context ( NGX_HTTP_LIF_CONF ) in ngx_http_config.h . Which implies defining a merge configuration function.
Document everything with Doxygen linking the relevant header and source files from the Nginx core.
Contributors 2
nginx module
We provide a nginx module, which exposes redirection.io right in your nginx configuration. Using this module is one of the advised ways of integrating redirection.io in your infrastructure .
We also provide an Apache module . Should your infrastructure use Apache, please rather use this module.
When an incoming HTTP(s) request arrives, it is handled by the nginx webserver, which redirection.io's native module hooks in, in a very fast and performant way:
Please note that the nginx module requires a redirection.io agent instance to be available on your infrastructure, so please setup the redirection.io agent first.
Installing the module
Debian and apt-based distributions.
If you use a Debian or APT-based Linux distribution and want to install libnginx-mod-redirectionio , please use the following commands:
Select your distribution
Install the apt-transport-https package:
Import our apt repository key:
Add our Debian repository url:
Update the apt cache and install redirection.io's nginx module:
Once the module is installed, restart the nginx service, for instance:
You're done! redirection.io is installed, up and running. You can go on with redirection.io's nginx configuration and should see some logs within minutes 🎉
Red Hat and RPM-based distributions
If you use a Red Hat/CentOS-like Linux distribution and want to install libnginx-mod-redirectionio , please use the following commands:
Import the signature key:
Add our RPM repository:
Install redirection.io's nginx module:
Ansible role
A redirectionio.nginx_module Ansible role is available . You may install it using Ansible Galaxy:
Basically, this role installs the redirection.io nginx module and makes sure you use the right repository locations, depending on your host operating system. It supports Debian and RHEL-based Linux distributions.
Configuration
Enabling the redirection.io is as simple as declaring the project key in the Virtualhost or configuration node where you want to enable redirection.io.
You can find the project key in the "instances" screen of the manager (click on the "Setup on your infrastructure" button).
For instance, this can be in you main server directive:
Or in a Virtualhost:
Or even in a Directory:
The redirectionio_project_key can either be double-quoted or not (eg., SOME_PROJECT_KEY_HERE or "SOME_PROJECT_KEY_HERE" ).
Module configuration directives
The libnginx-mod-redirectionio enables several configuration directives, which can be used wherever the redirectionio_project_key is used:
redirectionio
- Syntax: redirectionio on | off;
- Default: redirectionio off;
- Context: http, server, server if, location, location if
This directive enables or disables redirection.io for requests matching the current context. By default, redirection.io is disabled, but is automatically enabled when the directive redirectionio_project_key is set in a node of the configuration.
Using redirectionio off can hence be useful to disable redirection.io in a particular location:
redirectionio_project_key
- Syntax: redirectionio_project_key some_key;
- Default: none
Sets the redirection.io project key to use for requests matching the current context. When set, this directive automatically turns the redirectionio directive on.
redirectionio_logs
- Syntax: redirectionio_logs on | off;
- Default: redirectionio_logs on;
This directive enables or disables logs collection for the current matching context (however, please note that logs will only be sent if both the redirectionio directive and redirectionio_logs are set to on . In other terms, setting redirectionio_logs to on with redirectionio set to off won't collect any logs).
redirectionio_pass
- Syntax: redirectionio_pass ip:port | unix:///path;
- Default: redirectionio_pass 127.0.0.1:10301;
This directive specifies the location of the redirectionio-agent that the nginx module must use for matching requests. In a traditional setup, the agent is only available at 127.0.0.1:10301 . If you expose the agent through a file, then the value of the redirectionio_pass directive can, for instance, be of the form unix:///var/run/redirectionio.sock .
As of the release 2.2.0, this directive also allows to set some connection pool management options:
All the settings min_conns , keep_conns , max_conns and timeout are optional:
- min_conns (default 0 ): this is the minimal number of tcp connections that are permanently established between the nginx module and the redirection.io agent. If these connections do not exist, they are created, even if no HTTP request is handled by the nginx server.
- max_conns (default 10 ): this is the maximal number of tcp connections that can be established between the nginx module and the redirection.io agent at a given time. If all the connections are currently in used, the module will wait for the timeout duration, then log an error and let the request pass without handling it.
- keep_conns (default 10 ): this is the minimal number of tcp connections to be kept alive once they have been created.
- timeout (default 100 ): this is the duration, in milliseconds, while the nginx module will wait for a response from the agent. The redirection.io agent is very performant and, even with hundreds of thousands of rules, should always send a response much faster than this timeout value (usually in less than 1ms), but this setting ensures that, even in very hard load contexts, the agent call will not block the request for too long.
For example, with min_conns=1 keep_conns=10 max_conns=100 timeout=50 :
- 1 connection is established between nginx and the agent when nginx is started;
- nginx will create up to 100 connections to the agent (of course, it reuses available connections);
- if many connections were created and are no more useful, they will be released, but nginx will keep 10 connections alive;
- nginx will wait max. for 50 milliseconds for a response from the redirection.io agent.
redirectionio_scheme
- Syntax: redirectionio_scheme http | https;
- Available since: 2.0.0
This directive allows to force the scheme to use when matching the request. This is useful if you have defined rules using a URL trigger with an absolute URL value (containing a scheme, a domain name, etc.), but you want to use the same rules in a VirtualHost with another scheme.
Imagine, for example, a VirtualHost listening https traffic only, using a project in which rules are defined using the http:// scheme: URLs would never be matched, as no rule uses the https:// scheme. In this case, setting the redirectionio_scheme value to http will force the module to match with this scheme, not the real request scheme.
redirectionio_host
- Syntax: redirectionio_host some-host;
This directive allows to force the host to use when matching the request. This is useful if you have defined rules using URL triggers with absolute URL values (containing a scheme, a domain name, etc.), and your VirtualHost uses another hostname.
redirectionio_add_rule_ids_header
- Syntax: redirectionio_add_rule_ids_header on | off;
- Default: redirectionio_add_rule_ids_header off;
If this setting is enabled, a response header named X-RedirectionIo-RuleIds will be added to the response. Its value will contain the list of the redirection.io rule ids applied to this response, separated by the ; character.
redirectionio_set_header
- Syntax: redirectionio_set_header X-Custom-Header-Name HeaderValue;
- Available since: 2.2.0
This directive allows to pass the request to the redirection.io module by adding a request header filled with a given value. The value passed to the header can be a nginx variable, eg.:
This can be useful to pass variables from the nginx context to the redirection.io agent, that can be used in redirection rules using the request header trigger .
redirectionio_trusted_proxies
- Syntax: redirectionio_trusted_proxies 127.0.0.1,172.18.0.0/24;
- Available since: 2.3.0
This directive is used by the module to restrict which proxies are trusted before evaluating the X-Forwarded-* headers. This is in particular useful when using the IP Address trigger, in order to ensure that the evaluated IP address is valid and has not been forged.
The value to use for this directive is a comma-separated list of all the proxies IP addresses that should be trusted (it might be a CIDR notation for subnetworks).
The nginx module is opensource and available on our github account
Troubleshooting
The nginx reload / restart command does not work and errors unknown directive "redirectionio_project_key".
If nginx fails to reload and complains about an unknown directive redirectionio_project_key , this may indicate that the nginx module has not been correctly loaded in nginx.
Please check that your main nginx configuration file (usually /etc/nginx/nginx.conf ) contains a line to include enabled nginx modules (eg. include /etc/nginx/modules-enabled/*.conf; ). If this line is missing, you may force loading the module by adding the following line in /etc/nginx/nginx.conf :
Then, restart nginx (again), and it should be fine.
The module redirects to a wrong domain, or to a wrong port
In some production setups, the nginx server is not the first node of the infrastructure - there can be a Varnish, an Envoy proxy, a front nginx configured as a reverse proxy, etc.
By default, nginx will generate a redirection Location with the domain name and port number it received the request with. This means that, if the redirection.io nginx module runs in a nginx server listening in http (not https ) on the domain internal on port 8080 , the redirections Location header will be of the form Location: http://internal:8080 , which is obviously not what you want.
In those cases, you may add the absolute_redirect off; directive in the VirtualHost where the redirection.io module is enabled:
This will prevent nginx to rewrite/set the scheme, host and port in redirection targets using relative locations. This won't affect the redirection rules that you created to other external domains.
Dynamically choose the project key to use
You may need to manage several websites under the same nginx VirtualHost, while still wanting to keep separate redirection.io projects (for organizational reasons, to delegate the permissions to the right teams, etc.)
In this case, a first solution can be to use a nginx variable to dynamically define the redirection.io project key to use.
In the http section of the nginx configuration, define the $project_key variable:
And, in the VirtualHost, use this variable:
This setup allows to:
- map redirection.io projects to specific domains
- define a default redirection.io project to be used for all domains which are not overriden.
In other project configurations, you will still want to use several redirection.io projects, for permissions purpose, without the possibility to segment based on the domain name. It is the case, for example, if your website uses paths for geographical regions ( /uk/ , us , fr , etc.) and if you want to restrict the access of contributors to one region only.
In this case, using the $http_host nginx variable will not be convenient, and you will have to use the nginx location block:
If the VirtualHost already contained some directives in a / location, that you still want to use, then you have to replicate these directives in each of the location blocks.
Using the nginx module with a non-standard nginx install
Our nginx module is one of the recommended ways of integrating redirection.io in Web stacks. We distribute our nginx module for many distributions, which should cover most of the production cases.
However, it may happen that your hosting provider or provisioning solution installs a custom version of nginx, in non-standard paths or with incompatible dependencies.
This page will guide you through the steps to get our nginx module work with your custom nginx install, whatever its setup. The only requirement is the nginx version, which must at least be NGINX 1.9.11.
Download and build the libredirectionio library
The libredirectionio library is a Rust library that processes redirects and filtering from rules created using redirection.io. This library is used in our various proxies, and specifically in the nginx module.
Install Rust compilation tools:
Get the libredirectionio library sources:
And build the library:
If the compilation is successful (it might be long), you should get a target/release/libredirectionio.so static library.
Download and build the module source
The redirection.io's nginx module is available as an Opensource project in our github account .
First, install some compilation tools and utilities:
Retrieve redirection.io module sources ( pick the last release from our repository ):
Identify the Nginx version of your system:
This means that the nginx version is "1.18.0"
Get nginx sources for this version (of course, use the right version number, from the above command):
Then, build the module:
Install and load the module in nginx
Installing the module should be quite straightforward in most cases - you simply have to move the compiled file to the right folder.
Usually, the following command should show the nginx modules directory location:
Then, move the module in this folder (of course, use the directory name returned by the command above):
Last step, you now have to load the module in nginx - for this, add a load_module directive in nginx.conf , something like:
You may also want to add this line in a separate file, to keep track of your own changes. Most of the time, the main nginx configuration file includes module files (for example, include /etc/nginx/modules-enabled/*.conf; ) - so feel free to create your own /etc/nginx/modules-enabled/mod-http-redirectionio.conf config file.
Restart nginx, and you're done!
You can check that the module fully works by completing its configurations steps in the VirtualHost .
IMAGES
VIDEO
COMMENTS
NGINX modules consist of two parts; a config file and a module file. The config file tells NGINX how to compile the module and what it is called. The module
Nginx has a module chain. When Nginx needs to gzip or chunk-encode a response, it whips out a module to do the work. When Nginx blocks access to a resource
Wiki home · Converting Static Modules to Dynamic Modules · Compiling Modules · New Config Shell File · Old Config Shell File · Module API · API Usage Examples.
Example: A Simple “Hello World” Module · Step 1: Obtain the NGINX Open Source Release · Step 2: Obtain the Module Sources · Step 3: Compile the
I say "typically" because Nginx's module invocation is extremely customizable. It places a big burden on module writers to define exactly how
Building filter modules. Buffer reuse. Load balancing. Examples Code style ... Writing a C module ... The nginx string type ngx_str_t is defined as follows:.
Building the Module ... In the Nginx source, run configure , make , and make install . If you only want to build the modules and not the Nginx server itself, you
Configure the module. There's only one directive hello_world that is supported in the location context only. Example: location = /test { hello_world; }.
Most of the time, the main nginx configuration file includes module files (for example, include /etc/nginx/modules-enabled/*.conf; ) - so feel
... and when you need to customize NGINX processing – for example adding ... security – the NGINX JavaScript module (NJS) makes it easy.