Why is F5 ASM blocking CRLF characters before the request starts?

Introduction

An HTTP request is supposed to start with its request line. The very first bytes on the connection should be the method:

GET /account HTTP/1.1

When F5 ASM or Advanced WAF reports CRLF characters before request start, the request didn't start with GET or POST. It started with a blank line, a carriage return and line feed (written as CRLF or \r\n), and only then did the real request begin.

It looks like nothing. But where a request starts is something every device in the path has to agree on, and this is a case where they often don't.

Where to find it: Security › Application Security › Policy Building › Learning and Blocking Settings › HTTP protocol compliance failed. Settings shown are from my lab.

The Problem

CRLF is two bytes, 0x0D 0x0A. HTTP/1.1 uses them all the time. Every header line ends with one, and a blank line marks the end of the headers:

GET /account HTTP/1.1\r\n
Host: www.example.com\r\n
\r\n

All of that is normal. What this check is looking for is a CRLF in front of the request line:

\r\n
GET /account HTTP/1.1\r\n
Host: www.example.com\r\n
\r\n

The HTTP standard says a client must not send that. It also says, for robustness, that a server should ignore at least one blank line in front of a request. So some servers will happily accept it, and F5 chooses the stricter route and flags it.

The check is under Security › Application Security › Policy Building › Learning and Blocking Settings, inside HTTP protocol compliance failed, and it is enabled by default. As with all of these, the sub-violation is switched on by itself and blocking is set on the parent.

How to Recognise It

In the request log you will see HTTP protocol compliance failed, with the detail CRLF characters before request start.

The best evidence is a packet capture. If you run tcpdump on the BIG-IP and look at the start of the request, a normal request begins with the letters of the method. One of these begins with 0D 0A first:

0D 0A 47 45 54 20 2F (\r\n G E T space /)

Make sure you are looking at the start of the request and not the normal line endings inside it. People do confuse the two.

Then find out what sent it. Browsers don't do this. Where I have seen it, it is usually somebody's own code: a script that writes requests straight onto a socket, a monitoring tool with a badly written send string, or a custom proxy that leaves a stray line in its buffer between requests on a reused connection. Scanners also send it on purpose, to see which devices in the path tolerate it.

The annoying part with the home-made ones is that they work perfectly when pointed at the web server directly, because the web server tolerates the blank line. Then they fail as soon as they go through the WAF, and everybody blames the WAF.

The Risk

On its own a leading CRLF is not an exploit. It is not proof of request smuggling.

The risk is inconsistency. One device ignores the blank line and passes the request on, another rejects it with a 400, another processes it fine. When the devices in front of an application can't agree about where a request begins, logging becomes unreliable, troubleshooting gets confusing, and combined with other malformed tricks it is one of the ways parsers get desynchronised. That is what attackers are testing for when they send this.

At a security boundary I would always rather not rely on every device behind me making the same tolerant decision.

Mitigation

I would recommend leaving this enabled. A properly written client starts with the method, full stop. There is no application requirement for a blank line before GET.

When it fires, check the raw bytes and find the sender.

If it is a scanner, bot or unknown client, let it block.

If it is one of your own tools, fix the tool. Most of the time the answer is to stop building HTTP by hand and use a proper HTTP library, which won't make this mistake. If it is a monitor send string, check what is in front of the GET. If it is a custom proxy, look at how it handles the connection between one request and the next.

Please don't turn the check off for the whole policy because one internal script is broken. That removes it for every client on that application.

Gotchas and Limitations

This is not a check for CRLF anywhere in the request. HTTP needs CRLF at the end of every line. This only looks at CRLF before the method.

Don't confuse it with CRLF injection either. CRLF injection is about getting line breaks into header values or parameters to split headers or responses. That is a different problem with different protections. This one is purely about the very start of the message.

And because the HTTP standard tells servers to be tolerant here, you will find backends that accept these requests without complaint. That doesn't mean the request was right. It means the backend was being forgiving.

Takeaway

CRLF characters before request start means the client sent a blank line before the request line. Clients are not supposed to do that, even though some servers will let it go.

Leave it enabled, look at the first bytes of the request when it fires, and fix the tool or proxy that put the blank line there.