Home
Files2bz2
- Details
Another useful tool
This will read a directory and sub directories (if you choose to) and copy the files to another folder and each file will be bz2 compressed.This is useful for game servers where you have a fast download server.
The git can be found here https://github.com/mrosandich/files2bz2
Adding more space to your Ubuntu 14 on a VM server
- Details
But sometimes you need to extend the partition because something grows to big.
First thing you'll want to do is login to the VM and poweroff in the console (i used putty)
Second thing you will need to do is open VMware vSphere Client find the VM, right click and edit.
Find the drive device and give it a bigger size. I am going from 10GB to 40 GB.
Save your changes (click ok)
Right click the VM and power it on.
Open putty so you can log back into the VM. I login as root.
Now we will add a new partition:
type in: fdisk -l
look for something that looks like( it will have your hostname you used when you set it up):
Disk /dev/mapper/ZabbixServer--vg-root: 8329 MB, 8329887744 bytes
please note the volume name from the above text is: ZabbixServer-vg , yes it doesn't have 2 dashes (--)
You will also see a list of devices:
device boot , start , end /dev/sda1, 2048 ,499711
/dev/sda2, 501758, 20969471
/dev/sda5, 501760, 20969471
take note of the highest end block and add 1: 20969472
fdisk /dev/sda
Command (m for help): n
Command (m for help): p
Command (m for help): 3 (this will make /dev/sda3 it might be /dev/sda4 just remember for the extend step )
start block will be the highest end block
End block will be the default listed in my case: 104857599
Command (m for help):w
reboot the vm.
vgextend ZabbixServer-vg /dev/sda3 (sba3 is from above step)
vgdisplay ZabbixServer-vg | grep "Free"
show show something like: Free PE / Size 10240 / 40.00 GiB (if you see this then you got it right)
resize2fs /dev/ZabbixServer-vg/root
reboot
drive is now larger!
GeoIP Location
- Details
I am a fan of free data and sharing cool information
So I give props to maxmind.com for their free lite geo IP data base.The database can be found at : http://dev.maxmind.com/geoip/geoip2/geolite2/ They are CSV Files so you need to import them.
here is how I import them into MySQL:
Create the first table:
CREATE TABLE `geocityblocks` ( `network_start_ip` varchar(45) DEFAULT NULL, `network_prefix_length` bigint(20) DEFAULT NULL, `geoname_id` bigint(20) DEFAULT NULL, `registered_country_geoname_id` bigint(20) DEFAULT NULL, `represented_country_geoname_id` bigint(20) DEFAULT NULL, `postal_code` varchar(45) DEFAULT NULL, `latitude` float(10,6) DEFAULT NULL, `longitude` float(10,6) DEFAULT NULL, `is_anonymous_proxy` int(11) DEFAULT NULL, `is_satellite_provider` int(11) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 ;now import the very large 198meg CSV file. I run this command in MySQL Workbench:
load data local infile 'C:/Users/mell/Downloads/GeoLite2-City-CSV/GeoLite2-City-CSV_20141202/GeoLite2-City-Blocks.csv' into table `test`.`geoCityBlocks` fields terminated by ',' OPTIONALLY enclosed by '"' lines terminated by '\n' ignore 1 lines (network_start_ip,network_prefix_length,geoname_id,registered_country_geoname_id,represented_country_geoname_id, postal_code,latitude,longitude,is_anonymous_proxy,is_satellite_provider)
It should be around 3,080,877 rows
Now lets work on the other table:
CREATE TABLE `geocitylocations` ( `geoname_id` varchar(100) DEFAULT NULL, `continent_code` varchar(100) DEFAULT NULL, `continent_name` varchar(100) DEFAULT NULL, `country_iso_code` varchar(100) DEFAULT NULL, `country_name` varchar(100) DEFAULT NULL, `subdivision_iso_code` varchar(100) DEFAULT NULL, `subdivision_name` varchar(100) DEFAULT NULL, `city_name` varchar(100) DEFAULT NULL, `metro_code` varchar(100) DEFAULT NULL, `time_zone` varchar(100) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8
Now import the data to that table. I run this command in MySQL Workbench:
load data local infile 'C:/Users/mell/Downloads/GeoLite2-City-CSV/GeoLite2-City-CSV_20141202/GeoLite2-City-Locations.csv' into table `test`.`geoCityLocations` fields terminated by ',' OPTIONALLY enclosed by '"' lines terminated by '\n' ignore 1 lines (geoname_id,continent_code,continent_name,country_iso_code,country_name,subdivision_iso_code,subdivision_name, city_name,metro_code,time_zone)should be around 86,470 rows
for the US this is how an entry looks
geoname_id=4700168
continent_code=NA
continent_name=North America
country_iso_code=US
country_name=United States
subdivision_iso_code=TX
subdivision_name=Texas
city_name=Irving
metro_code=623
time_zone=America/Chicago
and finally some code to use the data:
PHP Code: https://github.com/maxmind/geoip-api-php/blob/master/src/geoip.inc
About the CSV files: http://dev.maxmind.com/geoip/geoip2/geoip2-csv-databases/
So lets make this stuff more useful. Since everything is stored in ipv6 as strings we need to add 2 more fields:
ipv6_binary as varbinary(16)
ipv6_binary_top as as varbinary(16)
So run this is MySQL workbench: its going to take a few minutes
ALTER TABLE `test`.`geocityblocks` ADD COLUMN `ipv6_binary` VARBINARY(16) NULL AFTER `is_satellite_provider` ; ALTER TABLE `test`.`geocityblocks` ADD COLUMN `ipv6_binary_top` VARBINARY(16) NULL AFTER `ipv6_binary` ;This is going to store the ipv6 address as binary in ipv6_binary. e.g. 4.28.169.0 = ffff 041c a900
Then we will compute the top range and add that as well in ipv6_binary_top:255 allowed address FFFF 041C A9FF
-- The below line to set the binary for all the address. it takes awhile to run and you will need a version of MySQL that supports INET6_ATON, (run time about 3-5 minutes) :)
update test.geoCityBlocks set ipv6_binary=INET6_ATON(network_start_ip);now increment the ipv6_binary_top: INET6_ATON(network_start_ip) + getIpRange(network_prefix_length)
You will need to use the function I made list at the bottom of the page.
matching the IP:
You will need to pad the hex up to 16:
select * from test.geoCityBlocks where ipv6_binary>=x'00000000000000000000ffff041ca900' && ipv6_binary_top<=x'00000000000000000000ffff041ca900'
Functions:
DELIMITER $$ CREATE FUNCTION `test`.`getIpRange` (network_prefix INT) RETURNS INTEGER BEGIN DECLARE x INT; DECLARE j INT; DECLARE y INT; SET x = 1; SET y = 1; SET j = 129-network_prefix; sloop: LOOP SET x = x + 1; IF x > j THEN LEAVE sloop; END IF; SET y = y + y; ITERATE sloop; END LOOP sloop; RETURN y; END
Google Client side Geocoding
- Details
Google does have a limit of 2000 address a day.
You could also hide the code to do geocoding of items that were missed by using a jquery or ajax:
<html> <head> <script type="text/javascript" src="http://maps.googleapis.com/maps/api/js?sensor=false"></script> <script> geocoder = new google.maps.Geocoder(); function DoAddress() { var InAddress = document.getElementById('address').value; var geocoder = new google.maps.Geocoder(); var results = geocoder.geocode ( { address: InAddress }, function callback(results) { document.getElementById('lat').value= results[0].geometry.location.lat(); document.getElementById('lng').value= results[0].geometry.location.lng(); } ); } </script> </head> <body> <br /> <input type="text" style="width:300px;" name="address" id="address" value="austin tx 78747" /> <input type="button" value="GeoCode" onclick="DoAddress()" /><br /><br /> Lat: <input type="text" style="width:200px;" name="lat" id="lat" value="" /><br /> Lng: <input type="text" style="width:200px;" name="lng" id="lng" value="" /><br /> </body> </html>
This is my version of hidding the geocoded address for bulk but on client side
This is PHP and Jquery.
Why did I do this? I have 90k address that need to be geocoded. I generally have 50 people working/using the site.
geocoder = new google.maps.Geocoder(); function DoAddress() { var InAddress = ''; var InId = ''; var geocoder = new google.maps.Geocoder(); var results = geocoder.geocode ( { address: InAddress }, function callback(results) { if(typeof(results[0]) != "undefined") { var mylat= results[0].geometry.location.lat(); var mylng = results[0].geometry.location.lng(); var pullurl = "/somefolder/updategeocode?i=" + InId + "&lng=" + mylng + "&lat=" + mylat + "&rn=" + Math.floor(Math.random() * 100000); //rn is just to keep browser fresh try { if (pullurl) { jQuery.get(pullurl, {}, function (data) { //data is your pages reponse. //we really don't care about it. //if valid geocode we have just updated the address //since our address our being pulled based on no lat,lng } , 'html'); } } catch (e) { } }//end undefined }//end call back ); } $(function (){ DoAddress();//Call this after page is loaded });
Page 10 of 40