Category Archives: python

How To Install Python 3 on CentOS 7

Subscribe to Darrenoneill.eu to get a free Linux Cheat Sheet! 

This tutorial shows you how to install Python 3, set up a local programming environment and create your first python program. The commands that you need to follow along with this tutorial are:

 

Getting your Server ready for the install:
sudo yum update
sudo yum install yum-utils
sudo yum groupinstall development
sudo yum -y install https://centos7.iuscommunity.org/ius-release.rpm

 

Installing Python and PIP
sudo yum -y install python36u
python3.6 -V
sudo yum -y install python36u-pip
sudo yum -y install python36u-devel

 

Setting up your local environment.
mkdir environments
cd environments
python3.6 -m venv my_env
source my_env/bin/activate

 

Creating your python program:Vi HelloWorld.py
print(“Hello, World!”)
HelloWorld.py

 

Get list of running instances in AWS emailed to you daily.

Say you share an AWS account with a few people and you’re the bill payer. You’d like to keep an eye on the running instances, but logging in each day to check can get tiring, especially if you are out and about and can’t get near a PC. A handy way to get the instances is to have a script running that gathers the instances and emails them to you. The below script does just that. Before you set up this script you will need set up AWS command line tools on your server. You can see how to to this in the blog post here.

 

 

 

#This is a bash script to get running instances from AWS and mail them on to the  relevent people

 

printf “Hi there,\n I am a bot so please don’t reply to me all I do is list  active EC2 instances in your AWS account! \n if my findings appear wrong please contact Darren In IT and he will ensure that they are corrected \n Running

 

Instances in EU-West-1 \n” > FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region eu-west- 1 >> FileToBeMailed.txt

 

printf “\n Running Instance in us-east-1\n” >>FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region us-east-1 >> FileToBeMailed.txt

 

printf “\n Running Instance in us-west-2\n” >> FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region us-west- 2 >> FileToBeMailed.txt

 

printf “\n Running Instance in ap-northeast-2\n” >> FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region ap-northeast-2 >> FileToBeMailed.txt

 

printf “\n Running Instance in ap-southeast-1\n” >> FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region ap-southeast-1 >> FileToBeMailed.txt

 

printf “\n Running Instance in ap-southeast-2 \n” >> FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region ap-southeast-2 >> FileToBeMailed.txt

 

printf “\n Running Instance in ap-northeast-1\n” >> FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region ap-northeast-1 >> FileToBeMailed.txt

 

printf “\n Running Instance in eu-central-1 \n” >> FileToBeMailed.txt

 

aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId]’ –filters Name=instance-state-name,Values=running –output text –region eu-central-1 >> FileToBeMailed.txt

 

printf “\n I will mail again tomorrow with am updated list of instances” >> FileToBeMailed.txt

 

 

First we start of by typing the text we want to head our email. Here we state that we are a bot and that if there are any issues contact Darren who created the bot. The  “ \n “ that you see telling the script to start a new line. We then state what will be the first region that we will get details for by writing “\n Running Instances in EU-West-1 \. Once we have finished this paragraph we direct the output to a file called FileToBeMailed.txt

 

Next we start by issuing our AWS commands, basically, what we are saying here is get all the running instances in eu-west-1 and then append them to the end of the FileToBeMailed.txt. We do this for each AWS region, each time appending to the end of the FileToBeMailed.txt. This builds up the list of running instances that we can see. When we have listed all the instances we print that we will mail again tomorrow with the instances.

 

Now that we have the file produced we just need to send the contents of this file to an email account of our choice. To do this we can use send mail and is discussed in the post here that details how to use send mail to send emails to your own account.

 

Producing HTML links from a CSV Using Python

Recently in work I had to generate a long list of html links from two columns that I had extracted from a MySQL database to a .csv file. The example below shows what steps are needed to generate html urls from a csv file that has the text of the link in one column and the url in the other. The columns that I extracted from the database into a .csv file looked like the following:

 

Master of Arts in Learning https://testurl.com/courseid=123
Postgraduate Diploma in Arts https://testurl.com/courseid=127
Postgraduate Diploma in Computing https://testurl.com/courseid=128

 

In the file, there were roughly 400 rows. I needed to add the name of the courses with their link to a website as a list. The list created would allow students to browse for their course and follow the correct link to their course.

 

I had access to the html so I needed to add roughly 400 html links. To do this manually would take hours. So I decided I would need a script to produce the html. The script I came up with looks like the below:

 

import csv, sys
sys.stdout = open(‘links.txt’ , ‘w’)
for row in csv.reader(open(‘courses.csv’),delimiter=”,”):
print (”'<a href =\””’+ row[1] + ”'”>”’ + row[0] + ‘ </a> </br>’)
sys.stdout.close()

 

  1. Line one in the script; Import csv, sys. imports modules that I needed. The csv module facilitates the opening of .csv files while the sys module allows the redirection of output to a file.
  2. The second line; sys.stdout = open(‘links.txt’ , ‘w’) Creates and opens a file in write mode called links.txt.
  3. Line three, is a for loop, that opens the courses.csv file and goes through it line by line.
  4. As the loop goes through the file line by line, it prints  <a href =” and then the contents of the second column (The URL). Then it prints the closing bracket for the tag. Next we print what is in column 1, the close </a> tag and finally a </br> tag which will insure that each link exists on a new line on our webpage.

 

When we run our script the output produced to the links.txt looks like:

 

<a href =”https://testurl.com/courseid=123″>  Master of Arts in Learning</a></br>
<a href =” https://testurl.com/courseid=127″> Post Graduate Diploma in Arts</a></br>
<a href =”https://testurl.com/courseid=128″> Post Graduate Diploma in Computing</a></br>

 

This code can then be copied to our website which will then display a list of links that users can select.

How to Tweet From Python

Step one

Download and install twython, using pip. I’m using python 3 so I typed the command

sudo pip3 install twython

 

 

Step 2

Go to apps.twitter.com and create an account using your Twitter username and password.

 

Step 3

Click ‘Create New App’

Step 4

Fill in the mandatory fields and click next, if you don’t have a website just pop in a placeholder.

 

Step 5

Once your app is created you will now will be brought through to a new page, click ‘Key and Access Tokens’.

 

Step 6

Take note of the Consumer Key (API Key) and Consumer Secret (API Secret)

 

Step 7

You will need to create an access token.

 

 

Step 8

You should now see your access tokens, take note of these also.

 

Step 9

Open up your favorite text editor, I’m using Idle, and enter the following:

from twython import Twython

 

APP_KEY = ‘YOUR_KEY_GOES_HERE’
APP_SECRET = ‘YOUR_SECRET_GOES_HERE’
OAUTH_TOKEN = ‘YOUR_TOKEN_GOES_HERE’
OAUTH_TOKEN_SECRET = ‘YOUR_TOKEN_SECRET_GOES_HERE’

 

twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
twitter.update_status(status=’this is a test tweet from a python script!’)

 

Step 7

Now run your script, if using Idle, you can do this by pressing F5.

 

Step 8

Check Twitter to make sure your tweet has sent.

Python: Searching CSVs and exporting rows based on string macthes

Recently I had a need to search through a CSV log file, that was thousands of rows long and find certain names that existed in the log file and copy these rows to another CSV. I had to do this for dozens of names, each of which could have dozens of matches. Instead of searching for each of these names and copying the rows to another CSV file (which would have taken hours!) I decided to write a script that would do this task in a matter of seconds:

The script below opens one CSV; the file which you want to search. It then saves a new file with the results. There is a list at the start of of the program called people. You can add your own strings here which you are looking for in your CSV file:

 

The Script


import csv

people = [‘Joe Bloggs’, ‘Tim O Reilly’, ‘Mary Bloggs’]

with open(‘users.csv’) as oldfile, open(‘report.csv’, ‘w’) as newfile:
for line in oldfile:
if any(people in line for people in people):
newfile.write(line)

Example

Say we have a log file with 1000s of rows and we want to find all activity for the users Joe Bloggs, Tim O’Reilly and Mary Bloggs. The log file may look something like this:

User Action
Joe Bloggs Login
Mary Burke login
Darren O Neill Updated post
Joe Bloggs Updated post
Tim O Reilly login
Mary Bloggs login

Now if we run the script we can see the file returned only contains the users we are interested in.

Joe Bloggs Login
Joe Bloggs Updated post
Tim O Reilly login
Mary Bloggs login