update cve year

 update  nvdcve set year = SUBSTRING(cve_id, 5,4)   ;

Serve Django3 Applications with Apache2.4 and mod_wsgi on Ubuntu 20

Software versions: here are my application versions:

Python 3.8.2
Django 3.1.1 
Apache/2.4.41 (Ubuntu)

Ubuntu 20.04.1 LTS

Installation:

Step 1: Install apache 2,  Django and other packages

sudo apt-get install libapache2-mod-wsgi-py3
sudo apt-get install python3-pip
sudo pip3 install Django
sudo pip3 install mysql-python

sudo apt-get install python3-mysqldb

Step 2: create Django project - I create project under /var/www, but you may put anywhere, it's up to you

django-admin.py startproject myproject

You will see these files and folder under /var/www/myproject

myproject/

django.wsgi*

manage.py*

Step 3: Create apache vhost file 

cat /etc/apache2/sites-available/xxxx.com.conf

<VirtualHost *:80>

    #My site Name

    ServerName xxxx.com

    DocumentRoot /var/www/myproject

    WSGIDaemonProcess xxxx.com processes=2 threads=5 display-name=%{GROUP} python-path=/usr/local/lib/python3.8

    #Pointing wsgi script to config file

    WSGIScriptAlias / /var/www/myproject/myproject/wsgi.py

    WSGIProcessGroup xxxx.com

    #Your static files location

    Alias /static/ "//var/www/myproject/static/"

    <Location "/media">

        SetHandler None

    </Location>

    <LocationMatch "\.(jpg|gif|png|js|css)$">

        SetHandler None

    </LocationMatch>

    <Directory /var/www/myproject >

        AllowOverride all

        Require all granted

        Options FollowSymlinks

    </Directory>

</VirtualHost>

Step 4: modify this file /var/www/myproject/myproject/wsgi.py

cat /var/www/myproject/myproject/wsgi.py

import os

import time

import traceback

import signal

import sys

from django.core.wsgi import get_wsgi_application

sys.path.append('/var/www/myproject')

sys.path.append('/usr/local/lib/python3.8/dist-packages')

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'CVE.settings')

try:

    application = get_wsgi_application()

except Exception:

    # Error loading applications

    if 'mod_wsgi' in sys.modules:

        traceback.print_exc()

        os.kill(os.getpid(), signal.SIGINT)

        time.sleep(2.5)

Step 5: enable wsgi mod and enable site

sudo a2enmod wsgi

sudo a2ensite xxxx.com

Summary: you may see these errors, please check RED part

mod_wsgi (pid=1059622): Failed to exec Python script file '/var/www/myproject/django.wsgi'.

 mod_wsgi (pid=1059622): Exception occurred processing WSGI script '/var/www/myproject/django.wsgi'.

 Traceback (most recent call last):

   File "/var/www/myproject/django.wsgi", line 12, in <module>

     application = django.core.handlers.wsgi.WSGIHandler()

   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/wsgi.py", line 127, in 

     self.load_middleware()

   File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 40, in 

     middleware = import_string(middleware_path)

   File "/usr/local/lib/python3.8/dist-packages/django/utils/module_loading.py", line 17, in 

     module = import_module(module_path)

   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module

     return _bootstrap._gcd_import(name[level:], package, level)

   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

   File "<frozen importlib._bootstrap>", line 991, in _find_and_load

   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

   File "<frozen importlib._bootstrap_external>", line 783, in exec_module

   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

   File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/middleware.py", line 3, in <

     from django.contrib.auth.backends import RemoteUserBackend

   File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/backends.py", line 2, in <

     from django.contrib.auth.models import Permission

   File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/models.py", line 2, in <

     from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager

   File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/base_user.py", line 48, in <

     class AbstractBaseUser(models.Model):

   File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 108, in __new__

     app_config = apps.get_containing_app_config(module)

   File "/usr/local/lib/python3.8/dist-packages/django/apps/registry.py", line 252, in 

     self.check_apps_ready()

   File "/usr/local/lib/python3.8/dist-packages/django/apps/registry.py", line 135, in 

     raise AppRegistryNotReady("Apps aren't loaded yet.")

 django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

less /var/lib/logrotate/logrotate.status

less /var/lib/logrotate/logrotate.status

MAC "fatal error: 'openssl/opensslconf.h' file not found"

Run these two command to fix it

export CPPFLAGS=-I/usr/local/opt/openssl/include

export LDFLAGS=-L/usr/local/opt/openssl/lib

corenlp

step 1: tokenize

  java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.process.PTBTokenizer 2-malware.txt > 2-malware.tok

  less 2-malware.tok

Step 2: mark:
  perl -ne 'chomp; print "$_\tO\n"' 2-malware.tok > 2-malware.tsv

Step 3: generate tsv file using customized KEYWORD file

generate final.tsv

Step 4: generate customized NER using malware.prop: => will create malware-ner-model.ser.gz

 java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.ie.crf.CRFClassifier -prop malware.prop

Step 5: using custom NER to detect text

java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.ie.crf.CRFClassifier -loadClassifier  malware-ner-model.ser.gz -testFile sample.tsv.ini


--------------------------------------
java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.process.PTBTokenizer jane-austen-emma-ch1.txt > jane-austen-emma-ch1.tok

java -mx15g -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.pipeline.StanfordCoreNLP -outputFormat json -file jane-austen-emma-ch1.txt

transmission 403 unauthorized ip address

When trying to access transmission from web-browswer i got the message :

403: Forbidden
Unauthorized IP Address.
Either disable the IP address whitelist or add your address to it.
If you're editing settings.json, see the 'rpc-whitelist' and 'rpc-whitelist-enabled' entries.
If you're still using ACLs, use a whitelist instead. See the transmission-daemon manpage for details.

I setup whitelist before and it was working fine. So, it is not setting.json issue. I check the log file:

root@ubuntu:~# less /var/log/syslog | grep trans
Jul 25 12:16:48 ubuntu transmission-daemon[26035]: [2017-07-25 12:16:48.151] Couldn't save temporary file "/home/debian-transmission/.config/transmission-daemon/settings.json.tmp.0uKjqQ": No such file or directory (variant.c:1280)
Jul 25 12:16:48 ubuntu transmission-daemon[26035]: Closing transmission session... done.
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] Couldn't create "/home/debian-transmission": Permission denied (file-posix.c:189)
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: message repeated 2 times: [ [2017-07-25 12:16:49.422] Couldn't create "/home/debian-transmission": Permission denied (file-posix.c:189)]
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] UDP Failed to set receive buffer: requested 4194304, got 425984 (tr-udp.c:84)
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] UDP Failed to set send buffer: requested 1048576, got 425984 (tr-udp.c:95)
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] Couldn't save temporary file "/home/debian-transmission/.config/transmission-daemon/settings.json.tmp.7GNxAD": No such file or directory (variant.c:1280)
Jul 25 12:38:04 ubuntu transmission-daemon[26365]: Closing transmission session... done.
Jul 25 12:38:10 ubuntu transmission-daemon[26698]: [2017-07-25 12:38:10.154] UDP Failed to set receive buffer: requested 4194304, got 425984 (tr-udp.c:84)
Jul 25 12:38:10 ubuntu transmission-daemon[26698]: [2017-07-25 12:38:10.154] UDP Failed to set send buffer: requested 1048576, got 425984 (tr-udp.c:95)
Jul 25 12:38:48 ubuntu transmission-daemon[26698]: Closing transmission session... done.

Somehow, the system looking for /home/debian-transmission/.config/transmission-daemon/settings.json, but my file is located: /etc/transmission-daemon/settings.json I am not sure how this happened, looks like it is transmission bug, but I fixed it:

service transmission-daemon stop
mkdir -p /home/debian-transmission/.config/transmission-daemon
chown debian-transmission:debian-transmission /home/debian-transmission/.config/transmission-daemon
cp /etc/transmission-daemon/settings.json /home/debian-transmission/.config/transmission-daemon/settings.json
service transmission-daemon start

push docker image to ecr

aws ecr create-repository --repository-name my_lamp docker tag my_lamp:latest 953643660.dkr.ecr.us-west-2.amazonaws.com/my_lamp:latest docker push 953643660.dkr.ecr.us-west-2.amazonaws.com/my_lamp:latest