Bot detection and custom bot lists

The tracker checks every click for signs of a bot. The built-in check always runs and cannot be configured, while your own bot lists add User-Agent fragments, IP addresses and subnets to it. Owners and admins can view and change bot lists.

What happens to a bot

The “bot” («бот») mark on its own does not block a click. Here is what it does:

• In a campaign with cloaking enabled, the bot is sent to the “white” — see Where filtered traffic goes.
• Without cloaking, the bot goes through the streams like a regular visitor. To keep bots out of a stream, add the Bot («Бот») = false condition to it — see Protecting a stream without cloaking.
• In reports, the click counts toward the Bots («Боты») metric, and the Bot grouping and filter separate such clicks from the rest.
• The {is_bot} macro returns 1 for a bot and 0 for other clicks. All macros are in the reference.

Built-in signs

The tracker checks the signs in the order listed in the table and stops at the first match. Bot lists are checked last: if a click has already been identified as a bot by a built-in sign, it keeps that sign's code, not custom_list.

The code is visible only in a campaign with cloaking enabled: in the click window, in the Cloaking («Клоака») section, the Reasons («Причины») line contains text like “bot/moderation by UA or connection (ua_signature)” («бот/модерация по UA или коннекту (ua_signature)»). How to open the click window and read the other reasons is covered on the Diagnostics: why a click went to “white” page.

VPN, Tor and proxy addresses are not counted as bots. They are filtered out by the Block proxy / VPN / Tor («Блокировать прокси / VPN / Tor») checkbox in cloaking or by the Proxy / DC («Прокси / ДЦ») = false condition in a stream.

How to add your own bot list

1. Click the Admin panel («Админ-панель») icon in the top right corner of the panel.
2. Choose Settings («Настройки»).
3. Go to the Bots and GEO («Боты и гео») tab. The bot list block is below the IP2Proxy block and starts with the text “Custom bot lists on top of the built-in detection” («Кастомные бот-листы поверх встроенного детекта»).
4. Paste User-Agent fragments into the Bot UA — substrings (one per line) («Бот UA — подстроки (по одной на строку)») field.
5. Paste addresses and subnets into the Bot IP / CIDR (one per line) («Бот IP / CIDR (по одному на строку)») field.
6. Click Save («Сохранить») under these fields.

The message “Saved — will be applied to the engine within ~15s” («Сохранено — применится к движку в течение ~15с») appears above the fields for three seconds. The list starts applying to new clicks within about 15 seconds; no restart is needed. Clicks already recorded are not recalculated.

To remove an entry, delete the line and click Save again. Empty fields mean that only the built-in check is running.

Filling-in rules

Wildcard masks like 203.0.113.* are not accepted in the IP field — write the subnet in CIDR format.

Both lists together must be smaller than 1 MiB. If they are larger, the message “JSON object expected” («Ожидается JSON объект») appears when you save.

There is one list for the whole installation, and it applies to clicks of all campaigns.

Checking the result

1. Wait 15 seconds after saving.
2. Open the campaign link from an IP address on the list or from a browser whose User-Agent contains a fragment from the list.
3. Open this campaign in the panel.
4. Click Statistics («Статистика»).
5. Go to the Click log («Лог кликов») tab.
6. Find your click. In the Bot column, it has the “bot” mark.

In a campaign with cloaking enabled, click the row of the click. In the Cloaking section, the Reasons line contains “bot/moderation by UA or connection (custom_list)” («бот/модерация по UA или коннекту (custom_list)»).

The stream selection check shows which stream a filtered visitor will get, without sending any traffic.

If it does not save

The error text appears above the fields. Both fields are saved together: if one of them has an error, neither is saved.

Invalid IP/CIDR («Некорректный IP/CIDR»)

The IP field has a line that cannot be read as an address or subnet. It is shown in the message after the colon. Fix or delete that line.

Bot list: maximum 10000 lines («Список ботов: максимум 10000 строк»)

One of the fields has more than 10000 lines. Replace individual addresses with subnets or remove unneeded entries.

Empty or too long bot list value («Пустое или слишком длинное значение бот-листа»)

One of the lines is longer than 512 bytes. Shorten it to a distinctive fragment.

Settings were changed in another tab. Refresh the page and repeat the edit. («Настройки изменены в другой вкладке. Обновите страницу и повторите правку.»)

While you were editing the list, the settings were saved by another admin or by you in another tab. After the page is refreshed, unsaved lines will be lost.

1. Copy your list from the fields.
2. Refresh the page.
3. Paste the list again.
4. Click Save.

Settings saved, but the engine has not confirmed applying them yet. Refresh the page and save again. («Настройки сохранены, но движок пока не подтвердил применение. Обновите страницу и повторите сохранение.»)

The list is written to the settings but is not yet applied to traffic.

1. Refresh the page.
2. Click Save under the bot list fields again.

If the message repeats, check the services on the System status page.