Docker Alpine+Python3+Django+uWGSI+Nginx

Django works great when you are running in local, but the moment you decide to move onto production, the headaches begin…

Since a few months ago I deploy everything using Docker, the result is something much cleaner and easier to maintain, so when moving my Django app to production, I choose Docker for it. As expected someone had already taken the time to build a Docker image that using ubuntu as is base image will run your Django site with uWGSI&Nginx, a marriage made in heaven.

My problem with that image is that it uses Ubuntu, don’t get me wrong, I love Ubuntu for something like my laptop, but for a Django app in an EC2 instance, it just feels too heavy. So I created an equivalent Dockerfile, this time with the much lighter Alpine.

The Dockerfile is available in Github with instructions on how to deploy your app.
Enjoy!

Redshift UDF Phone Number to Country

Redshift’s UDFs (User Defined Functions) permit to execute, with some limitations, certain Python libraries and custom code.

In my case, I wanted to find a way to extract the country code from a phone number in E.164 format. UDFs are a perfect fit for this, the implementation in SQL would most certainly require creating custom views, and hacking your way around, while in python, we can use the library phone-iso3166

>>> from phone_iso3166.country import *
>>> phone_country('+1 202-111-1111')
'US'
>>> phone_country('+34645678901')
'ES'

To upload a library to Redshift, we first need to check it follows the structure:

directory/
    __init__.py
    extra_files.py
    subdirectory/
        __init__.py
        other_files.py

In our case, phone-iso3166 is already in that structure. Now we need to zip the library:

tar -xvzf phone-iso3166.tar.gz
zip -r phone-iso3166.zip phone-iso3166

With our zipped library, we need to upload it to S3. I did this part manually into a bucket named s3://redshift/custom-udfs/

Now, connect to Redshift and issue:

CREATE LIBRARY phone_iso3166 LANGUAGE plpythonu 
FROM 's3://redshift/custom-udfs/phone_iso3166.zip' 
CREDENTIALS 'aws_access_key_id=<your-aws-key>;aws_secret_access_key=<your-aws-pass>' 
region as '<your-region>';

Last step:

CREATE OR REPLACE FUNCTION udf_phone_country (phone_number VARCHAR(64)) RETURNS VARCHAR(64) IMMUTABLE as $$ 
from phone_iso3166.country import phone_country, InvalidPhone
try:
    return phone_country(phone_number)
except:
    return None
$$ LANGUAGE plpythonu;

You should be all set to use this in your queries:

SELECT 
analytics.udf_phone_country('+14151111975'),
analytics.udf_phone_country('+7652112311'),
analytics.udf_phone_country('+34626472918')

That returns:

"US","RU","ES"

 

DD-WRT Remote SSH Access behind VPN

SSH access doesn’t work when OpenVPN client is enabled on DD-WRT.
Packages do arrive at the router if you try to SSH against the WAN IP, however, because all OUTPUT  traffic is diverted through the VPN (interface tun0) SSH won’t succeed.

What’s missing is an OUTPUT rule on iptables to route traffic on port 22 through the vlan2 interface (that’s the interface connected directly to the internet).

First, create table 202 via the Gateway Ip on the Interface VLAN2:

$ ip route add default via $(nvram get wan_gateway) dev vlan2 table 202

Then apply the rule on table 202 to packages marked with 22.

$ ip rule add fwmark 22 table 202

Finally, tag with 22 every output package on port 22 not coming from any machine on the local network.

$ iptables -t mangle -I OUTPUT -p tcp --sport 22 -d ! 192.168.1.0/24 -j MARK --set-mark 22

Note that the last command skips packages from the local network in my case 192.168.1.0/24, reason being that when SSHing from a host in local, the packages should be routed through br0 and not vlan2.

First issue these commands in the command line of your router to ensure they work with you, if somehow they break your routing, a restart will clear them. Once you have made sure they work, you can add them to the firewall script of your router

Note that my config IP and port is different because I am not using the default values.

 

Redshift Force Drop User

Ever tried dropping a user in Redshift only to discover that user “user_1” cannot be dropped because the user has a privilege on some object.

That’s not of great help Redshift. Luckily for us, there is a solution. Kamlesh Gallani posted a response about revoking permissions for tables and schemas that I user might still be assigned to, along with changing the ownership of tables that the user might have created. After that, dropping the user is straightforward.

I created a python snippet with his response that might save you a few minutes. You will need the view v_get_obj_priv_by_user from amazon-utils, simply create the view on your Redshift cluster, then, copy and paste this Python script into your favorite editor (e.g. Sublime Text) fill in the connection information and the list of users to drop and execute it.

import psycopg2
 
# Connect to Redshift
conn = psycopg2.connect(host="", dbname="", user="", password="", port="5439")
conn.autocommit = True
cursor = conn.cursor()
 
# List of users to drop
users = ['user_to_drop_1', 'user_to_drop_2']
 
# New owner, used when changing ownership
new_owner = 'new_user'
 
# Templates to change ownership, revoke permissions, and drop users
change_ownership = "select 'alter table '+schemaname+'.'+tablename+' owner to %s;' from pg_tables where tableowner like '%s'"
revoke_schema_permissions = "select distinct 'revoke all on schema '+schemaname+' from %s;' from admin.v_get_obj_priv_by_user where usename like '%s'"
revoke_table_permissions = "select distinct 'revoke all on all tables in schema '+schemaname+' from %s;' from admin.v_get_obj_priv_by_user where usename like '%s'"
drop_user = "drop user %s;"
 
for user in users:
    # Change ownership
    cursor.execute(change_ownership % (new_owner, user))
    for r in cursor.fetchall():
        print("Executing: %s" % r[0])
        cursor.execute(r[0])
    # Revoke schema permissions
    cursor.execute(revoke_schema_permissions % (user, user))
    for r in cursor.fetchall():
        print("Executing: %s" % r[0])
        cursor.execute(r[0])
    # Revoke table permissions
    cursor.execute(revoke_table_permissions % (user, user))
    for r in cursor.fetchall():
        print("Executing: %s" % r[0])
        cursor.execute(r[0])
    # Drop user
    cursor.execute(drop_user % (user))

Miele W 1065 – Wiring for 110 Volts

Miele W 1065 Back

I got yesterday one of these Miele beauties. She is over 20 years old, and my guess is that she weights around 250 pounds. Yes, it was though carrying it downstairs from is previous owner to home with just 2 guys. 

When I brought her home, I saw that it required 220V which it’s a problem where I live, BUT if you are willing to sacrifice the second heater,  you can adapt can change the wires to make it work with the regular 110V.

Miele W 1065 Back
Miele W 1065 Back

First and foremost unplug the washer, unscrew the white plastic box and that will expose the connections box. You will see we have Ground, Neutral, phase 2 and phase 1 when reading from right to left. The washer will work for with just having phase 2 connected to 110V, but as said above, the heater won’t work, so you are stuck with washing with cold water. Which anyways is the best way to preserve your clothes.

I bought a piece of cord and cut it to expose the wiring, then just connect ground to ground, neutral to neutral and phase 2 to the remaining. I used a multimeter to make sure the wiring was correct.

Try this at your own risk. Mine is working like a champ.

WordPress HTTP Error when uploading files

Today I got this error when uploading a video to my WordPress site. I Googled around for a little bit, but the proposed solution wasn’t working.

It turns out that since I am using WordPress docker and a Nginx proxy, this later was complaining that the file was too large 413 Request Entity Too Large when the file was uploaded using the browser uploader, not the WordPress multi-file uploader.

Wordpress Browser Uploader
WordPress Browser Uploader

The solution was easy:
$ vim nginx.conf

http {
...
# Added because uploading large files to WordPress is throwing HTTP error.
client_max_body_size 64M;
...

In my case since I want my setting to take place for all the websites I am hosting with Nginx I applied the setting under the http section.

Enjoy

SET Game Visual Solver

For those of you that have never played SET® before, SET® is a fast-paced game involving 81 unique cards with different combinations of four characteristics (color [red, green, or purple], shape [diamond, oval, or squiggle], shading [hollow, shaded, or filled], and number [1, 2, or 3]). The point of the game is to find as many 3-card “sets” as possible. If the three cards are either all the same or all different in each of the four characteristics, you know you’ve found a set. Twelve cards are presented on the table at a time, and are continually replenished when a set is taken out, until no cards remain in the deck. Sometimes, however, there are no sets in the twelve cards, in which case 3 more cards are dealt. But, how do you know if there really is no set? I aim to solve this problem using image processing software and eventually create a set recognition mobile application.

So, what determines a set? Let’s take the example below (although it is probably one of the most difficult sets in the game). Looking at the cards in terms of the four characteristics, we have the following: color – red, green, and purple (all different); shape – diamond, oval, and squiggle (all different); shading – filled, hollow, shaded (all different); number – 1, 2, 3 (all different). If the cards were exactly the same, but all red, this would also be considered a set, because, in terms of color, they are all the same.

 

There’s the simplified breakdown of how I used an image processing algorithm to determine sets from ordinary jpeg images:

Step 1: Convert the image to black and white and invert the image to calculate the number of shapes per card

Step 2: Compare the pixel surface area of one of the symbols on each card to determine the shape

Step 3: Normalize the RGB color of the images with an equation to figure out the color

Step 4: Convert the images to gray scale and apply a border detection to determine the shading

Step 5: Run the simple algorithm that deduces the sets

For more details, check out the paper I wrote.

BlackJack Hitting vs. Standing chances

Who has never thought about beating the house in a Casino? As the MIT students said in their documentary Breaking Vegas, it’s not all about the money; but about the feeling of beating a huge corporation in their own game.

It’s very easy to find online the basic strategy for BlackJack, i.e., the correct decision when the only information that you have about the game are your cards, and the dealers facing card. Understanding by correct decision, the one that would minimize your losing chances. Even with this basic strategy the odds against the house are negative, but greatly reduced to a mere -3 $ per 100$ played (aprox). Of course the Breaking Vegas students didn’t just played basic strategy, they also counted cards and played in a team not to raise any attention from the Big Brother.

For this post, I wrote a small Python notebook that computes what are the chances for the different combinations of cards and strategies in their most simplified way, standing and hitting.

The full notebook: https://github.com/Koff/blackjack/blob/master/blackjack_simulation.ipynb

And the results in two images:

 

On the Y-Axis, the starting score of the player, and on the X-Axis, the dealer’s facing up card (11 being the Ace). The color and values represent the chances of winning if the following the strategy.

[Android] Baldur’s Gate II Text Issue

I recently purchased Baldur’s Gate II on Google Play Store, but just after a few minutes of playing, I noticed how small the UI elements and texts were displayed on my LG G3.

Furthermore turning on the setting UI Scaling doesn’t change anything for screens above Full-HD, which is the case of the LG G3.
I decided to then open the file Baldur.ini located in:

/sdcard/Android/data/com.beamdog.baldursgatellenhancededition/files

With Root Explorer (you don’t need a rooted device). Then go to the lines 75 & 76 and change them from:

'Fonts', 'Zoom', '148',
'Fonts', 'Size', '3',

to:

'Fonts', 'Zoom', '248',
'Fonts', 'Size', '6',

If you would also like to increase the upper limit for the FPS by default capped at 30 fps, go to line 70 and change it, note that this can drain your battery:

'Program Options', 'Maximum Frame Rate', '30',

to:

'Program Options', 'Maximum Frame Rate', '60',

My final Baldurs.ini looks like this:

CREATE TABLE options (
section string,
name string,
value string
);
INSERT INTO options ROWS (
'Fonts', 'ko_KR', 'UNBOM',
'Fonts', 'zh_CN', 'SIMSUN',
'Fonts', 'ja_JP', 'MSGOTHIC',
'Fonts', 'ru_RU', 'PERMIAN',
'Fonts', 'uk_UA', 'PERMIAN',
'Graphics', 'version', 'OpenGL ES 3.0 V@84.0 AU@ (CL@) - build 207',
'Graphics', 'renderer', 'Adreno (TM) 330',
'Graphics', 'vendor', 'Qualcomm',
'MOVIES', 'LOGO', '1',
'Graphics', 'Scale UI', '1',
'Game Options', 'Footsteps', '1',
'Game Options', 'Memory Level', '1',
'Game Options', 'Mouse Scroll Speed', '36',
'Game Options', 'GUI Feedback Level', '5',
'Game Options', 'Locator Feedback Level', '3',
'Game Options', 'Bored Timeout', '3000',
'Game Options', 'Always Dither', '1',
'Game Options', 'Subtitles', '1',
'Game Options', 'Keyboard Scroll Speed', '36',
'Game Options', 'Command Sounds Frequency', '2',
'Game Options', 'Selection Sounds Frequency', '3',
'Game Options', 'Effect Text Level', '62',
'Game Options', 'Infravision', '0',
'Game Options', 'Weather', '1',
'Game Options', 'Tutorial State', '1',
'Game Options', 'Attack Sounds', '1',
'Game Options', 'Auto Pause State', '0',
'Game Options', 'Auto Pause Center', '1',
'Game Options', 'Difficulty Level', '3',
'Game Options', 'Suppress Extra Difficulty Damage', '0',
'Game Options', 'Quick Item Mapping', '1',
'Game Options', 'Environmental Audio', '1',
'Game Options', 'Heal Party on Rest', '1',
'Game Options', 'Terrain Hugging', '0',
'Game Options', 'HP Over Head', '0',
'Game Options', 'Critical Hit Screen Shake', '1',
'Game Options', 'Hotkeys On Tooltips', '1',
'Game Options', 'Area Effects Density', '100',
'Game Options', 'Duplicate Floating Text', '1',
'Game Options', 'Tiles Precache Percent', '100',
'Game Options', 'Color Circles', '1',
'Graphics', 'Zoom Lock', '0',
'Game Options', 'Over Confirm Everything', '0',
'Game Options', 'Show Learnable Spells', '1',
'Game Options', 'Render Actions', '2',
'Game Options', 'Confirm Dialog', '1',
'Multiplayer', 'Disable Banters', '1',
'Program Options', 'Disable Cosmetic Attacks', '0',
'Game Options', 'Render Travel Regions', '1',
'Game Options', 'Pausing Map', '0',
'Game Options', 'Extra Feedback', '0',
'Game Options', 'Filter Games', '1',
'Game Options', 'All Learn Spell Info', '0',
'Graphics', 'Hardware Mouse Cursor', '1',
'Game Options', 'Maximum HP', '1',
'Game Options', 'Show Character HP', '1',
'Game Options', 'Nightmare Mode', '0',
'Game Options', '3E Thief Sneak Attack', '0',
'Game Options', 'Cleric Ranger Spells', '1',
'Program Options', 'Font Name', '',
'Program Options', 'Double Byte Character Support', '0',
'Program Options', 'Drop Capitals', '1',
'Program Options', '3D Acceleration', '1',
'Program Options', 'Maximum Frame Rate', '60',
'Program Options', 'Path Search Nodes', '32000',
'Program Options', 'Tooltips', '2147483647',
'Program Options', 'Translucent Shadows', '1',
'Program Options', 'Sprite Mirror', '0',
'Fonts', 'Zoom', '248',
'Fonts', 'Size', '6',
'Program Options', 'Volume Movie', '90',
'Program Options', 'Volume Music', '40',
'Program Options', 'Volume Voices', '100',
'Program Options', 'Volume Ambients', '40',
'Program Options', 'Volume SFX', '80',
'MOVIES', 'INTRO15F', '1',
'MOVIES', 'INTRO', '1',
'Multiplayer', 'Last Protocol Used', '1',
'Game Options', 'Last Save SOA', '000000001-Quick-Save',
'MOVIES', 'BLACKPIT', '1',
'Game Options', 'Last Save TOB', '000000004-Quick-Save-TOB',
'MOVIES', 'REST', '1',
'Window', 'Maximized', '0',
'MOVIES', 'DEATHAND', '1',
'MOVIES', 'POCKETZZ', '1',
'MOVIES', 'SARADUSH', '1',
'MOVIES', 'RESTDUNG', '1'
);

APAC Country List

This is probably the most stupid post of this website up to the moment, but I couldn’t find the information anywhere else; so here it is, the list of country codes for the Asia Pacific region (APAC) and it’s two letters ISO equivalent.

Australia,AU
Brunei,BN
Cambodia,KH
People's Republic of China,CN
Hong Kong,HK
Macau,MO
Fiji,FJ
Indonesia,ID
Japan,JP
Kiribati,KI
North Korea,KP
South Korea,KR
Laos,LA
Malaysia,MY
Marshall Islands,MH
Federated States of Micronesia,FM
Nauru,NR
New Zealand,NZ
Palau,PW
Papua New Guinea,PG
Philippines,PH
Samoa,AS
Singapore,SG
Solomon Islands,SB
Thailand,TH
Timor-Leste,TP
Tonga,TO
Republic of China (Taiwan),TW
Tuvalu,TV
Vanuatu,VU
Vietnam,VT

Maybe it will save someone’s 5 mins.