corenlp

step 1: tokenize

  java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.process.PTBTokenizer 2-malware.txt > 2-malware.tok

  less 2-malware.tok

Step 2: mark:
  perl -ne 'chomp; print "$_\tO\n"' 2-malware.tok > 2-malware.tsv

Step 3: generate tsv file using customized KEYWORD file

generate final.tsv

Step 4: generate customized NER using malware.prop: => will create malware-ner-model.ser.gz

 java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.ie.crf.CRFClassifier -prop malware.prop

Step 5: using custom NER to detect text

java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.ie.crf.CRFClassifier -loadClassifier  malware-ner-model.ser.gz -testFile sample.tsv.ini


--------------------------------------
java -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.process.PTBTokenizer jane-austen-emma-ch1.txt > jane-austen-emma-ch1.tok

java -mx15g -cp "stanford-corenlp-full-2017-06-09/*" edu.stanford.nlp.pipeline.StanfordCoreNLP -outputFormat json -file jane-austen-emma-ch1.txt

transmission 403 unauthorized ip address

When trying to access transmission from web-browswer i got the message :

403: Forbidden
Unauthorized IP Address.
Either disable the IP address whitelist or add your address to it.
If you're editing settings.json, see the 'rpc-whitelist' and 'rpc-whitelist-enabled' entries.
If you're still using ACLs, use a whitelist instead. See the transmission-daemon manpage for details.

I setup whitelist before and it was working fine. So, it is not setting.json issue. I check the log file:

root@ubuntu:~# less /var/log/syslog | grep trans
Jul 25 12:16:48 ubuntu transmission-daemon[26035]: [2017-07-25 12:16:48.151] Couldn't save temporary file "/home/debian-transmission/.config/transmission-daemon/settings.json.tmp.0uKjqQ": No such file or directory (variant.c:1280)
Jul 25 12:16:48 ubuntu transmission-daemon[26035]: Closing transmission session... done.
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] Couldn't create "/home/debian-transmission": Permission denied (file-posix.c:189)
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: message repeated 2 times: [ [2017-07-25 12:16:49.422] Couldn't create "/home/debian-transmission": Permission denied (file-posix.c:189)]
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] UDP Failed to set receive buffer: requested 4194304, got 425984 (tr-udp.c:84)
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] UDP Failed to set send buffer: requested 1048576, got 425984 (tr-udp.c:95)
Jul 25 12:16:49 ubuntu transmission-daemon[26365]: [2017-07-25 12:16:49.422] Couldn't save temporary file "/home/debian-transmission/.config/transmission-daemon/settings.json.tmp.7GNxAD": No such file or directory (variant.c:1280)
Jul 25 12:38:04 ubuntu transmission-daemon[26365]: Closing transmission session... done.
Jul 25 12:38:10 ubuntu transmission-daemon[26698]: [2017-07-25 12:38:10.154] UDP Failed to set receive buffer: requested 4194304, got 425984 (tr-udp.c:84)
Jul 25 12:38:10 ubuntu transmission-daemon[26698]: [2017-07-25 12:38:10.154] UDP Failed to set send buffer: requested 1048576, got 425984 (tr-udp.c:95)
Jul 25 12:38:48 ubuntu transmission-daemon[26698]: Closing transmission session... done.

Somehow, the system looking for /home/debian-transmission/.config/transmission-daemon/settings.json, but my file is located: /etc/transmission-daemon/settings.json I am not sure how this happened, looks like it is transmission bug, but I fixed it:

service transmission-daemon stop
mkdir -p /home/debian-transmission/.config/transmission-daemon
chown debian-transmission:debian-transmission /home/debian-transmission/.config/transmission-daemon
cp /etc/transmission-daemon/settings.json /home/debian-transmission/.config/transmission-daemon/settings.json
service transmission-daemon start

push docker image to ecr

aws ecr create-repository --repository-name my_lamp docker tag my_lamp:latest 953643660.dkr.ecr.us-west-2.amazonaws.com/my_lamp:latest docker push 953643660.dkr.ecr.us-west-2.amazonaws.com/my_lamp:latest

update ruby version

cng@ACP:/opt/newdisk/mydockers/docker-lamp$ source /usr/local/rvm/scripts/rvm cng@ACP:/opt/newdisk/mydockers/docker-lamp$ rvm use 2.4.1 Using /home/chang/.rvm/gems/ruby-2.4.1 cng@ACP:/opt/newdisk/mydockers/docker-lamp$ ruby -v ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

chef 12 opsworks github issue

I had some issue to checkout git repo from private repo.
Took couple of hours to figure it out.

1: add application cookbook in "Berksfile"
cookbook 'application',             '~> 5.2.0'

2: run 

berks update

3: create recipe:

app = search(:aws_opsworks_app).first

app_path = "/srv/#{app['shortname']}"

application app_path do

  git  app['app_source']['url'] do

    deploy_key  app['app_source']['ssh_key']

  end

end

rails binding to all ip

rails server --binding=0.0.0.0

rails server --binding=0.0.0.0 -d

ROR Dockerfile

FROM ubuntu:16.04

# Set the working directory to /app
WORKDIR /ROR

# Copy the current directory contents into the container at /app
ADD . /ROR

# Install any needed packages specified in requirements.txt
RUN \
  apt-get update && \
  apt-get -y upgrade && \
  apt-get install -y git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev nodejs  && \
  mkdir /ROR && \
  cd /ROR && \
  git clone https://github.com/rbenv/rbenv.git ~/.rbenv && \
  echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc && \
  echo 'eval "$(rbenv init -)"' >> ~/.bashrc && \
  exec $SHELL \ &&
  git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build && \
  echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc && \
  exec $SHELL && \
  rbenv install 2.4.0 && \
  rbenv global 2.4.0 && \
  ruby -v

oozie enable extjs

  327  locate core-site.xml

  330  vim /usr/local/hadoop/etc/hadoop/core-site.xml

<property>
<name>hadoop.proxyuser.hduser.hosts</name>
<value>localhost</value>
</property>
<property>
<name>hadoop.proxyuser.hduser.groups</name>
<value>hadoop</value>
</property>

  331  bin/oozie-setup.sh prepare-war
  332  bin/oozied.sh stop
  333  bin/oozie-setup.sh prepare-war
  334  bin/oozied.sh start

clean cache

sync; sudo echo 3 > /proc/sys/vm/drop_caches