AuditpolCIS – Automating Windows SIEM CIS Benchmarks Testing

In the previous post on the subject of Windows SIEM, we covered the CIS benchmarks for Windows Auditing Policy in a spreadsheet, which was provided freely (really, actually free).

This week we introduce a python open source tool we have developed, to automate the CIS Benchmark testing.

Download AuditpolCIS

Meeting Regulatory / Compliance / Audit Requirements

The automated assessment results from AuditpolCIS, as it’s based on CIS Benchmarks, helps in the support of meeting audit requirements for a number of programs, not least PCI-DSS:

  • Audit account logon events: Helps in monitoring and logging all attempts to authenticate user credentials (PCI-DSS Requirement 10.2.4).
  • Audit object access: Monitors access to objects like files, folders, and registry keys that store cardholder data (PCI-DSS Requirement 10.2.1).
  • Audit privilege use: Logs any event where a user exercises a user right or privilege (PCI-DSS Requirement 10.2.2).
  • Local log files sizes and retention policies are useful in assessing compliance with e.g. 5.3.4 and 10.5.1 requirements (PCI-DSS 4). There should be a block of text after the audit policy results.

Usage / Setup

First you will to set up a Python Virtual Environment. Ensure that you have Python installed on your system (Python 3.10 was used in development). If not, download and install Python from the official website: https://www.python.org/downloads/

Open a Command Prompt or terminal window and navigate to the folder where you extracted the AuditpolCIS project.

Run the following command to create a new virtual environment:

python -m venv venv

Activate the virtual environment by running:

For Windows:

venv\Scripts\activate

For macOS/Linux:

source venv/bin/activate

Install the required Python packages from the requirements.txt file by running:

pip install -r requirements.txt

You will need a .env file in your project root. The contents relate to the target you wish to test:

HOSTNAME='<Windows box IP address or host name>'
USERNAME='<Windows user account name>'
PASSWORD='<account password>'


Make sure to assign the right ownership and permission on .env. Usually the permissions will be 600.

Once the virtualenv is enabled, you can run the code:

./auditpolcis.py


Feel free to branch or submit a PR.

Additional Points

The CIS benchmarks are based on Windows 2019 Server but they apply to other target varients on a Windows theme. I know none of you will have EOL Windows versions. <Sarcasm engaged>I mean in 22 years of consulting, i’ve never seen any out-of-support warez in critical business usage</Sarcasm engaged>.

Powershell is not required on the target but use of Powershell is also not a crime. Yes, that was a security person who said that.

Sustainability / Use of Regex

I had to use some fairly snazzy regex to pull out Categories (category_pattern = r'^(\w+.*?)(\r)?$') and Subcategories (subcategory_pattern = r'^( {2})([^ ]+.*?)(?=\s{3,})(.*\S)') from the auditpol command output. I did look at more sustainable ways of achieving the same goal, although admittedly i didn’t spend much time doing that. One thing has been clear for a long time with Windows – don’t go looking for registry keys because that can be very painful. Not only is documentation for a key location somewhat thin and erroneous, the key loation also often changes across Windows versions. ChatGPT‘s lack of knowledge of Windows reg keys bears testimony to the previous comments.

So there are two sources of Subcategory names – there is cis-benchmarks.yaml and there is the output of the auditpol /get /category:* command. If there are entries in the YAML file which are not in the auditpol output, they are flagged in the script output, and the same is true vice versa. So if you make spelling mistakes in the excel sheet or YAML file, it will be flagged. It can also happen that auditpol output subcategories do not reflect the CIS Benchmarks subcategories, perhaps with different Windows versions as targets. Any of these categories will be flagged by the script and listed below the pass/fail results.

If you want to change the verdicts or [Sub]Category names, you are of course free to do so. You can edit the cis-benchmarks.yaml file, or edit the included spreadsheet, followed by running the included genyaml.py.

Connection Method

The scripts works over SSH because other types of connection are a pain in the derriere and require you to radically increase your attack surface area, but if there’s a request for e.g. WinRM, please do let me know, or send out a Pull Request. Follow this link for more information about enabling the built-in SSH for Windows.

I know use of AutoAddPolicy with Paramiko in Python is not good form, but also assume that as an admin in the position of someone who performs daily tasks using administrative rights, that you know your hosts. Sometimes security people do get in the way of progress, when there’s low risk issues afoot. Use of RejectPolicy instead of auto-add would be one such case.

Tests Rationalisation

Some of the tests included are not a CIS Benchmark (out of 59 tests, 32 are CIS Benchmarks, whereas 27 are not). It’s not clear why the subcategories were omitted by CIS but anyway – in these cases we have made an assessment based on logging events volume for this subcategory, versus the security value of the subcategory. Most of these are just noise, and in many cases, very high volume noise, so we have advised “No Auditing”.

Customising Test Criteria

The testing template is formed of the YAML file cis-benchmarks.yaml. If you prefer to make changes to the testing template with Excel, the sheet is CIS-Audit-Reqs-Windows2019Server.xlsx in the code root. You can then use the python script genyaml.py to generate a new YAML file (you will need to use the right virtualenv, see above for usage instructions).

Django and Celery – Two Sites, Single Host

Documenting this here because Celery’s documentation isn’t the best in general, but moreover because I hadn’t seen a write-up for this scenario – which I would imagine is not an uncommon situation.

So here’s the summarised scenario:

The challenge was to have 2+ Django sites on one VM without confusing the Celery backend. This requires the creation of a “Queue”, “Exchange”, and “Routing Key”. The Celery documentation gives major clues but doesn’t cover Django to any large degree. It does cover some first steps with Django, but nothing about deploying custom Queues in terms of Django python files and what goes where, or at least i couldn’t find it.

I’m sure there are many ways of achieving the same result but this is what worked for me…

The application won’t be able to find celery if you don’t initiate it. The project here is ‘netdelta’. Under the <app> dir create a file, say celery_app.py:

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from kombu import Exchange, Queue



# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'netdelta.settings')

app = Celery('nd')

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings')

# Load task modules from all registered Django app configs.
#app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

app.conf.task_queues = (
    Queue('cooler',  Exchange('cooler'),   routing_key='cooler'),
)
app.conf.task_default_queue = 'cooler'
app.conf.task_default_exchange_type = 'direct'
app.conf.task_default_routing_key = 'cooler'

@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

Which is called when the project fires up with use of __init.py__ under <app> (‘nd’ in this case):

from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery_app import app as celery_thang

__all__ = ('celery_thang',)

<django root>/<proj>/<proj>/settings.py is where the magic happens and it turns out to be a few lines. ‘scheduled_scan’ is the <proj>.tasks.<celery_job_name>:

CELERY_QUEUES = {"coller": {"exchange": "cooler",
                              "routing_key": "cooler"}}

CELERY_ROUTES = {
    'nd.tasks.scheduled_scan': {'queue': "cooler",
                                'exchange': "cooler",
                                'routing_key': "cooler"},
}

Now do the same for the other site(s).

Launch a worker thusly …

python manage.py celery worker -Q cooler -n cooler --loglevel=info -B
  • ‘cooler’ is the site name
  • ‘nd’ is the app name

The final step – your app.task has to call the custom queue you defined. In my case i could afford to set the default QUEUE to my wanted QUEUE because i had need for only one queue. But in multiple QUEUE scenarios, you will need to define the queue. I was setting jobs to run under a schedule using the djcelery admin panel that was created by default. The result are database entries – using an ‘INSERT’ statement to show the table structure (The queue, exchange, and routing key are highlighted):

INSERT INTO `djcelery_periodictask` (`id`, `name`, `task`, `args`, 
`kwargs`, `queue`, `exchange`, `routing_key`, `expires`, `enabled`, 
`last_run_at`, `total_run_count`, `date_changed`, `description`, 
`crontab_id`, `interval_id`) VALUES
(2, 'Linode-67257', 'nd.tasks.scheduled_scan', '["Linode"]', '{}', 
'coller', 'coller', 'coller', NULL, 1, 
'2017-10-15 23:40:00', 18, '2017-10-15 23:41:29', '', 2, NULL);

 

My virtualenv is as below (note i’m using the fork of libnmap that doesn’t use multiprocessor:

>pip list

  • amqp (2.2.2)
  • anyjson (0.3.3)
  • appdirs (1.4.3)
  • billiard (3.5.0.3)
  • celery (4.1.0)
  • Django (1.11.6)
  • django-celery-beat (1.0.1)
  • django-celery-results (1.0.1)
  • django-dajaxice (0.7)
  • html2text (2017.10.4)
  • iptools (0.6.1)
  • kombu (4.1.0)
  • MySQL-python (1.2.5)
  • netaddr (0.7.19)
  • packaging (16.8)
  • pip (9.0.1)
  • pyparsing (2.2.0)
  • python-libnmap (0.7.0)
  • pytz (2017.2)
  • setuptools (36.6.0)
  • six (1.11.0)
  • vine (1.1.4)
  • wheel (0.30.0)