Python Zappa + django-cookiecutter (via PyCharm)

A brief guide on deploying an django-cookiecutter baked app to AWS Lambda via Python Zappa. Using PyCharm. Greetings to my future self using this guide in several years.

Cookie-cutter, PyCharm

Install cookiecutter

pip install "cookiecutter>=1.4.0"

Lets create our project with cookiecutter-django (it will create a directory within the directory you are currently in of the same name as ‘project slug’). N.b. Sentry is autoselected to be installed, which can be very useful for debugging in Zappa (n.b one pitfall is that Sentry needs to access services outside of AWS, and the process of granting Zappa external internet access can be troublesome – a catch-22 scenario. For this reason, it is very helpful, be it very briefly, to have debug=True when you first get your app online via Zappa).

 

cookiecutter https://github.com/pydanny/cookiecutter-django

Lets open the project in pycharm.

 

Create virtual environment via File -> Settings… select the cog on the far right side of the window, under ‘Reset’ .

We need to tell Pycharm where our requirement files are (select ‘…’ button top right).

Select production.txt.

 

You need to open a file in PyCharm, and magically, it asks to install packages missing from your virtual environment.

While packages are being installed, let’s create a local run configuration so we can verify the project is working locally. Let’s clean up the existing configurations too (down arrow, ‘Edit Configurations…’). I’m removing all the Docker configs.

 

Note that in ‘Environment variables’ you need to add DJANGO_SETTINGS_MODULE=config.settings.local

 

Zappa

We need to set up our AWS credentials. Head over to https://pypi.org/project/awscli/ for proper instructions. You can just follow the below instructions once you’ve set up your amazon account (and have created access and secret keys):

 

pip install awscli
aws configure
AWS Access Key ID: foo
AWS Secret Access Key: bar
Default region name [us-west-2]: us-west-2
Default output format [None]: json

Let’s now install zappa and apply it to our project.

pip install zappa$
zappa init

 

In PyCharm, open zappa_settings.json (press shift key twice and start typing 😉 ).

This is very minimal! I stole the below from one my projects. Feel free to copy/paste your settings into mine below. Note that we will set up ‘vpc config’ stuff in another blog post. If you do not remove this text now (the crossed out text), zappa will fail when you try to deploy.

{
    "dev": {
      "environment_variables": {
        "SECRET_KEY": ---",
        "RDS_PASSWORD": "---",
        "RDS_INSTANCE_ID": "---",
        "RDS_DB_NAME": "---",
        "RDS_USERNAME": "---",
        "RDS_HOSTNAME": "---",
        "DJANGO_SETTINGS_MODULE": "config.settings.production",
        "DJANGO_SECRET_KEY": "---",
        "AWS_ACCESS_KEY_ID": "---",
        "AWS_SECRET_ACCESS_KEY": "---",
        "REDIS_LOCATION": "redis://---.---.cache.amazonaws.com"
       },
      "aws_region": "eu-west-1",
      "django_settings": "config.settings.production",
      "profile_name": "spyre",
      "project_name": "---",
      "runtime": "python3.6",
      "s3_bucket": "---",
      "exclude": [
        "*.pyc",
        "*.sqlite3",
        "*.mp4",
        ".ebextensions",
        ".idea",
        "static",
        "my_staticfiles"
      ],
      "cors": true,
      "slim_handler": false,
      "certificate_arn": "arn:aws:acm:us-east-1:---:certificate/---",
      "keep_warm": true,
      "vpc_config": {
           
"SubnetIds": [
             
"subnet----"
             
],
           
"SecurityGroupIds": [ "sg----" ]
        }
,

      "extra_permissions": [{
            "Effect": "Allow",
            "Action": ["ses:SendRawEmail", "ses:SendEmail", "ses:SendTemplatedEmail"],
            "Resource": "*"
        }]
    }
}

In order to access system variables within zappa_settings.json on your local machine we need the following at the top of base.py

 

import os ON_DEV_SERVER = False
if 'SERVERTYPE' not in os.environ or os.environ['SERVERTYPE'] != 'AWS Lambda':
    ON_DEV_SERVER = True
    import json
    import os

    json_data = open('zappa_settings.json')
    env_vars = json.load(json_data)['production']['environment_variables']
    for key, val in env_vars.items():
        if key not in ['DJANGO_SETTINGS_MODULE', 'django_settings']:
            os.environ[key] = val

 

In local.py let’s quickly add db info


DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'zappademo',
        'USER': 'zappademo',
        'PASSWORD': 'zappademo',
        'HOST': '127.0.0.1',
        'PORT': '5432',
    }
}

 

Let’s migrate our db

python manage.py migrate

and run our local server

With luck we get:

Unfortunately, Zappa wants your apps to be in the top directory, whilst Django-bootstrap has them all in zappa_demo (here). Just move them to your top directory. You’ll need to update various files within your project to reflect this directory change. An easy way of doing this is via control+shift+f to search for ‘zappa_demo’ and then just remove the defunct directory info.

 

I found I had to adjust remove the below ‘zappa_demo’ text that’s been labelled with a blue line.

With luck, your app still works on your local machine.

One last thing we need to do is remove the Argon2 password hash. I’ve never managed to get it to work in Lamba, and neither has PyDanny (reddit link; the author of Django-cookiecutter, and fellow keen zappa-nought).

Deploying Zappa

I'd advise briefly having debug=True in production (but at a very obscure aws provided url) whilst you get your app up and running on Zappa. Elsewise, it's next to impossible to figure out what's going on with any arising bugs. 

Once run, this can take a few minutes. You run this command each time you want to update your site. The first deploy takes a little longer than the others as several packages need to be downloaded.

I got the below once, the site was launched and I visited the deployment url.

Running zappa tail, reveals my problem, I’ve not set up the db! Setting this up and accessing the external internet from the app is the focus of the next blog.

This article was updated on 4 May 2018

Comments