Posts

Showing posts from May, 2016

How to install python2.7 with pip2.7

Python is one of the most famous and powerful languages used so in order to install or update it to 2.7 version following steps can be used.


Installing the Python2.7.
[root@ip-10-0-1-55 ~]# yum install python27 python27-devel[root@ip-10-0-1-55 ~]# python --version Python 2.7.10
If its still showing older version add the new version as default.
[root@ip-10-0-1-55 ~]# alternatives --config python There are 2 programs which provide 'python'. + 1 /usr/bin/python2.6 * 2 /usr/bin/python2.7 Enter to keep the current selection[+], or type selection number: 2 [root@ip-10-0-1-55 ~]# python --version Python 2.7.10
Installing the pip2.7.
[root@ip-10-0-1-55 ~]# wget https://bootstrap.pypa.io/get-pip.py --2016-05-26 11:10:33-- https://bootstrap.pypa.io/get-pip.py Resolving bootstrap.pypa.io (bootstrap.pypa.io)... 103.245.222.175 Connecting to bootstrap.pypa.io (bootstrap.pypa.io)|103.245.222.175|:443... connected. HTTP request sent, awaiting response... 200 OK Length:…

How to create and remove swap partition in Linux

If our server is having memory issues and we want to increase it without updating the physical RAM than Swap is a very good option. It's slower than the physical RAM but can do the tasks also if created on faster hard disks the results are good.

Create a Swap partition in place where you wanna create it. I am creating it in /swap with 2GB of size.
[root@ip-10-0-1-38 /]# dd if=/dev/zero of=/swap bs=1M count=2048 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB) copied, 54.8004 s, 39.2 MB/s
Update the permissions of /swap partition to 600.
[root@ip-10-0-1-38 /]# swapon /swap swapon: /swap: insecure permissions 0644, 0600 suggested. [root@ip-10-0-1-38 /]# chmod 600 /swap
Start the Swap on /swap partition.
[root@ip-10-0-1-38 /]# swapon /swap
To check if swap has started or not.
[root@ip-10-0-1-38 /]# swapon -s Filename Type Size Used Priority /swap file 2097148 0 -1 [root@ip-10-0-1…

How to install SyntaxNet in Linux

SyntaxNet is an open-source neural network framework for TensorFlow that provides a foundation for Natural Language Understanding (NLU) systems.
In order to install it in Linux servers following steps can be used..

It requires Python 2.7 so if you don't have that install the same.
[root@ip-10-0-1-55 ~]# yum install python27 python27-devel[root@ip-10-0-1-55 ~]# python --version Python 2.7.10
If its still showing older version add the new version as default.
[root@ip-10-0-1-55 ~]# alternatives --config python There are 2 programs which provide 'python'. + 1 /usr/bin/python2.6 * 2 /usr/bin/python2.7 Enter to keep the current selection[+], or type selection number: 2 [root@ip-10-0-1-55 ~]# python --version Python 2.7.10
Install the java1.8 version.
[root@ip-10-0-1-55 ~]# yum install java-1.8.0-openjdk*
If old java version is showing up update it via following command.
[root@ip-10-0-1-55 ~]# alternatives --config java
Make sure that your java home is pointi…

Take dump of all the databases from mysql server

To take MySQL dump of all the databases following script can be used it will work for normal MySQL server as well as Amazon RDS.
[root@ip-10-0-1-231 ravi]# cat dump.sh #!/bin/bash #Script to get dump of all the databases within the server. USER="root" PASSWORD="dbpassword" databases=`mysql -h prod-XXXXXXXXXXXX.rds.amazonaws.com -u $USER -p$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database` for db in $databases; do if [[ "$db" != "information_schema" ]] && [[ "$db" != "performance_schema" ]] && [[ "$db" != "mysql" ]] && [[ "$db" != _* ]] ; then echo "Dumping database: $db" mysqldump -h prod-XXXXXXXXXXXXXX.rds.amazonaws.com -u $USER -p$PASSWORD --databases $db > `date +%Y%m%d`.$db.sql # gzip $OUTPUT/`date +%Y%m%d`.$db.sql fi done

How to add numbers via bash

There are cases when we need to add up numbers or the outputs of our commands via bash so in that case following commands are helpful.

[ec2-user@ip-10-0-1-38 ~]$ cat list 45 78 56 67 34 56
To do the sum:
[ec2-user@ip-10-0-1-38 ~]$ cat list | awk '{ SUM += $1} END { print SUM }' 336
Same can be used to get sum of any output for example grep.
[ec2-user@ip-10-0-1-38 ~]$ cat s3data.txt | grep Size | grep Mi | awk '{ print $3 }' | awk '{ SUM += $1} END { print SUM }' 3405





Find size of S3 Buckets

In order to find the size of S3 buckets we can use following ways:

First Method: via s3api cli
[root@ip-10-0-1-231 ravi]# aws s3api list-objects --bucket bucketname --output json --query "[sum(Contents[].Size), length(Contents[])]" [ 30864102, 608 ]
30864102: Is the size in Bytes.
608: No of objects in bucket.

Second Method : via s3 cli
[root@ip-10-0-1-231 ravi]# aws s3 ls s3://bucketname --recursive | grep -v -E "(Bucket: |Prefix: |LastWriteTime|^$|--)" | awk 'BEGIN {total=0}{total+=$3}END{print total/1024/1024" MB"}' 29.4343 MB
Using bash commands to get output in desired way.

Third Method : via s3 cli with parameters
[root@ip-10-0-1-231 ravi]# aws s3 ls s3://bucketname --recursive --human-readable --summarize 2016-05-04 11:32:00 7.6 KiB prompthooks0.py Total Objects: 1 Total Size: 7.6 KiB
-human-readable: this will provide the data that is already in KB, MB, GB. TB etc.

To get size of all the buckets in your S3 use the following…