Posted on Leave a comment

Amazon Lightsail

Amazon Lightsail is a VPS services offered by Amazon that competes with the likes of Rackspace, DigitalOcean, Linode, etc. Note: As of mid 2018 AWS effectively halved its prices on Lightsail. This means there is a $2.50 USD/mo. option and the $40 option listed below (4gb ram/2 cpu/60gb ssd/4tb xfer) is actually only $20 now. Compared head-to-head the Lightsail option is a middle-of-the-road offering. However, compared with AWS and including the highly optimized nature of running Amazon Linux AMI (and not overselling with bullshit numbers like some providers), Amazon Lightsail is an extremely attractive VPS. S3 snapshot backups and other aspects of high reliability make this a go-to package for the VPS market.

Lightsail Specifications

See the Amazon Lightsail FAQs - Up to 3 DNS zones - Up to 5 ip addresses (elastic IPs) - Available in about half of AWS zones The various sizes of Lightsail are (as of July 2017): - $5/mo. - 512mb ram, 1 core, 20gb ssd, 1tb transfer - $10/mo. - 1gb ram, 1 core, 30gb ssd, 2tb transfer - $20/mo. - 2gb ram, 1 core, 40gb ssd, 3tb transfer - $40/mo. - 4gb ram, 2 core, 60gb ssd, 4tb transfer - $80/mo. - 8gb ram, 2 core, 80gb ssd, 5tb transfer Note that transfer allowances are half of the above, for Mumbai and Sydney currently.

Lightsail vs. EC2 Pricing

The real genius in Lightsail is the pricing. Compared with a 1 year reserved T2.Nano instance, a $5 Lightsail would be as follows: - T2.Nano reserved $3.36/mo - 1gb data xfer, subsequent at $0.09/gb/mo. (1tb = $89.91) - 1 elastic ip vs. 5 elastic ips (instances can only use 1 in ec2) - $0.11gb/mo of provisioned ebs (disk) = $2.22/mo in value - DNS = 3 zones included vs. $0.50/mo/zone for Route53 Total value of $8.13-$98.04 in value (depending mainly on data transfer). However, if you had only a single zone, a single IP, 8gb of disk (smallest available), and under 1gb of data transfer, then the value is $4.74/mo., which is within 5% of the cost of a $5/mo. Lightsail. That said, it is not clear how the vcpu works under Lightsail vs. EC2. However, since this is a single infrastructure, likely the performance is similar, and AWS is just going after a different segment of the market (one that is price-conscious).

Lightsail Docs and CLI

Lightsail has docs and a cli.

Lightsail Tasks

  • Create zone(s)
  • Create and download SSH Cert
  • Log in from command prompt with
    • ssh -i /path/to/.ssh/key.pem ec2-user@server.domain.tld
  • Operate under root rights with sudo su

Lightsail Control Panel

Lightsail is not integrated into the rest of AWS, though it is possible to see some aspects of it (perhaps storage?) from the console. Definitely it is managed separately from EC2 and Route53. This lack of integration is a bit of a pain, but likely it will go away (slowly and partially) over time (perhaps).

Securing Lightsail

Depending upon one's security requirements, it might be useful to create a new user and disable or remove rights to the ec2-user account. The steps to create a user with the same rights as ec2-user are: - create the account useradd username - set a password for the account passwd username - add the account to the sudo group usermod -aG wheel username - log in with the account su - username - create a .ssh directory mkdir .ssh - set security on the directory chmod 700 .ssh - log out of username exit - now back in root, copy the authorized_keys file to username

cp /home/ec2-user/.ssh/authorized_keys /home/username/.ssh/authorized_keys

Log all the way out of the system, and try and log in with the username, and same public key. Once logged in invoke sudo su to ensure it has the correct rights. There should be an error message. The last step is to replace ec2-user with username in the file: /etc/sudoers.d/cloud-init If this works, then you have a new account with the same priviledges as the ec2-user (and you have also removed ec2-user from the ability to become root) and can safely delete (or ignore) that account.

Lightsail Limitations

Lightsail has a few limitations, including no tools for transfer or resizing, though in late 2018 an ability to export snapshots to EC2 was added. In addition, Lightsail cannot port filter at the IP address, only at the port level. And for DNS management, CAA records are not supported (as opposed to Route 53 where they are).

Posted on Leave a comment

AMI on EC2 vs. CentOS on Linode

What I learned in migrating from CentOS on Linode to Amazon AMI Linux on EC2. Note: Amazon Lightsail is probably a better comparison, but it is not available in the region we need it in, so EC2 is required for now. Update: Lightsail is now available in more regions.

VPS Hosting and Operating System

A change of VPS providers is generally not such a big deal, unless one is dependent on proprietary tools of the previous provider. In the case of Linode, there are some nice graphs, and they have the Linode Shell (Lish) that is handy when things go south. But moving to Amazon is like entering a different galaxy, one that has alien technology. Sure, in the end you've got a web server, an email server, a vpn, etc., but configuration tools and where things go can be a bit of a learning curve.

Linode to AWS

I've been aware of Linode for a decade, but first started seriously kickin the tires in April, 2014. In November, it was time and I began installation, configuration, and testing, with production sites up in early December, 2014. At 2.5 years, this is the approximate time span for moving to a different platform (in terms of my Sysops habits). Actually, though, I have been content with Linode. The precipitating event to migration was a requirement for a location in Canada, and also an unacceptable amount of downtime on a different VPS host located there. Taking a closer look at AWS offerings, and especially getting a better understanding of billing, convinced me this is viable platform for my price/performance needs.

CentOS vs. AMI

Amazon Linux AMI is more or less a fork of RHEL and/or CentOS, depending on whom one reads. The main points are: - AMI lags a bit behind CentOS, which lags a bit behind Fedora - AMI may or may not yet have systemd (as of June 2017 it does not, treat the system as CentOS 6.x) - firewalld may or may not work on AMI, some reports are it does not yet (more on this) - systemctl does not function out of the box, instead use service and chkconfig

Firewalling

AWS Security Groups as configured in EC2 are managed in a GUI that is akin to iptables/firewalld.

IP Addressing

Unlike most VPS hosts, public IP addresses are not doled out automatically, at least not permanent ones. For that, one needs to provision an elastic ip address. A small number are available per account per region (5), but there are charges for outbound traffic. Also, it is important not to have elastic ip addresses assigned to unactive instances, as there are additional charges, as well as if multiple elastic ip addresses are assigned to a single instance. - EC2 Instance IP Addresses - VPC IP Addressing Guide

EB vs. EC2 vs. Lambda

There are other options to explore with AWS (just to mention): - Elastic Beanstalk, which is kind of self-provisioning (but need stateless design) - Lambda which is a truly serverless environment - Note that Lambda can now run on IOT as Greengrass Core with Lambda Functions

AWS Billing

It is vital to understand how Amazon bills out various services. Not paying attention can easily mean a lot higher costs, without necessarily more services. Another aspect of billing is the apparent greater efficiency of Amazon Linux AMI on EC2, which means that greater resources are not needed (and as is the case with much of the rest of the hosting universe, those numbers that are provided are not real, due to oversubscription.

Posted on Leave a comment

OpenVPN on Amazon Linux

OpenVPN on an AWS EC2 T2.Nano Instance

The T2.Nano instance is the smallest instance generally available for AWS EC2. As of 17-June-2017, the Nano includes the following resources: - 512mb RAM - 1 vcpu (30 credits + 3/hr, up to 72 credits) - 1gb network out traffic Alternatively, a $5 USD Amazon Lightsail instance can be used (see below)

Amazon Linux AMI

For those who prefer RHEL/CentOS, these are not available for the T2.Nano instance, rather Amazon Linux AMI is the only RHEL-derrived OS available. Note that Amazon AMI Linux is akin to CentOS 6.x (no systemd). Alternatively, Ubuntu is also available for the Nano. Note, there is now (Dec 2017) an Amazon Linux 2 option. Some say not to use any Amazon Linux. I tend to agree, though the main reason of not being able to use AMI outside of EC2 isn't correct, as there are container versions available for use locally.

Amazon Lightsail as an Alternative to EC2 T2.Nano

Amazon Lightsail is a VPS package that provides simplified control panel, and greater resources. For $5 USD/month, the smallest Lightsail instance is essentially a T2.Nano plus Elastic IP address, 20gb EBS storage, 1tb of outbound data, and Route53 DNS interface. Since outbound data can run 0.10/gb (with elastic IP), this is potentially $10/mo in database. The EBS storage is ~$2 USD, Route53 is $0.50 USD, and a nano instance with 1 year contract is ~$3.50 USD. This means for $5 USD/mo, one gets between $6-106 USD in AWS resources. For the $10 USD Lightsail, the value consists of a T2.Micro, and all the rest, which is worth $11-$211 USD in services due to an increase to 30gb EBS and 2TB data transfer out. - Amazon Lightsail FAQ Note: on Lightsail, the Security Groups are port-based only, so any IP filtering needs to be done wiht a separate firewall, such as iptables.

Steps to install OpenVPN on AMI - Pre-Installation

These steps are similar for a Nano instance. This should work on a Lightsail instance, though some control panel settings may be in different places.

Assume Root

sudo su

Set the hostname, timezone, nameservers

hostname server.domain.tld

Set the timezone

nano /etc/sysconfig/clock

Change the ZONE line to appropriate continent/city, e.g.,

ZONE="Continent/City"
UTC=false
ARC=false

Create a symbolic link

rm -rf /etc/localtime
ln -sf /usr/share/zoneinfo/Continent/City /etc/localtime

Update nameservers (using dns.watch resolvers)

echo "nameserver 84.200.69.80" > /etc/resolv.conf
echo "nameserver 84.200.70.40" >> /etc/resolv.conf

Edit the network sysconfig

nano /etc/sysconfig/network

Change HOSTNAME to server.domain.tld Check to ensure the change with the command:

hostname

Don't worry about /etc/hosts for now...

reboot

Update yum, configure EPEL

Note that we want the Amazon EPEL Repository

yum clean all
yum update
yum -y install epel-release
yum -y install yum-utils
yum-config-manager --enable epel

Update AMI without EPEL

This is done by disabling the repositories, which can be enabled later, including:

yum-config-manager --disable epel
yum clean all
yum update
cat /etc/system-release
uname -r

After the update version is confirmed, then re-enable the repositories with:

yum-config-manager --enable epel

Secure SSHD

nano /etc/ssh/sshd_config

make sure of the following:

PasswordAuthentication no
PermitRootLogin no

If you want to do fancy stuff like have an sftp login inside of a web directory, and need different than 700, 750, or 755 rights (say, for example, having the group be apache, and the user be a login) then include:

StrictModes no

Restart sshd

service sshd restart

Install and enable MOSH

yum -y install mosh

Mosh makes connections more resilient, but there is a cost of disabling the ability to scroll up in the console.

firewalld or ufw

This may or may not be desirable, in addition to the AWS firewall configuration. Likely desirable.

Install OpenVPN on AMI

yum -y install openvpn

Install Easy-RSA on AMI

Note that since there is a version 3.x, have to get an older distribution or it won't work. Note the below might still be a bit of a mess. Inspect directories as you go.

cd /etc/openvpn
wget -v https://github.com/OpenVPN/easy-rsa/releases/download/2.2.2/EasyRSA-2.2.2.tgz
tar -xvzf EasyRSA-2.2.2.tgz
mkdir easy-rsa
mv EasyRSA-2.2.2 /etc/openvpn/easy-rsa
cd easy-rsa
mv EasyRSA-2.2.2 2.0
mkdir -p /etc/openvpn/easy-rsa/keys
cp -R /usr/share/easy-rsa/2.0/ /etc/openvpn/easy-rsa/

NAT routing using iptables

Put in nat routing, ensure that the network on the masquarade is the same as in /etc/openvpn/server.conf First edit the iptables-config file

nano /etc/sysconfig/iptables-config

Change most things to yes, with a final config looking like:

IPTABLES_MODULES=""
IPTABLES_MODULES_UNLOAD="yes"
IPTABLES_SAVE_ON_STOP="yes"
IPTABLES_SAVE_ON_RESTART="yes"
IPTABLES_SAVE_COUNTER="no"
IPTABLES_STATUS_NUMERIC="yes"
IPTABLES_STATUS_VERBOSE="yes"
IPTABLES_STATUS_LINENUMBERS="yes"

Now do the rest of the iptables configuration

touch /etc/sysconfig/iptables
chkconfig iptables on
service iptables start
modprobe iptable_nat
echo 1 | tee /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -o eth0 -s 10.8.0.0/24 -j MASQUERADE
service iptables save
service iptables restart

Edit the Easy RSA settings

nano /etc/openvpn/easy-rsa/2.0/vars

Find and modify these values:

# These are the default values for fields
# which will be placed in the certificate.
# Don't leave any of these fields blank.
export KEY_COUNTRY="CA"
export KEY_PROVINCE=""
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"
export KEY_OU="MyOrganizationalUnit"

Also change

export KEY_CONFIG=`$EASY_RSA/whichopensslcnf $EASY_RSA`

to

export KEY_CONFIG=/etc/openvpn/easy-rsa/2.0/openssl-1.0.0.cnf

Initialize Easy RSA and create Certs and Keys

cd /etc/openvpn/easy-rsa/2.0
chmod 0755 *
source ./vars
./clean-all
./build-ca

Verify success

ls -la keys

Now build the cert and key

./build-key-server server

Note: leave the challenge password and optional company name blank Next, Verify success

ls -la keys

Next build a cert and key for each vpn user:

./build-key username

Provide this with a challenge password Next, build the .pem

./build-dh

Next, build the ta.key // rather forget about this, just comment out, it is trouble // get this going later openvpn --genkey --secret /etc/openvpn/easy-rsa/2.0/keys/ta.key Copy the keys and certs

cd /etc/openvpn/easy-rsa/2.0/keys
cp dh2048.pem ca.crt server.crt server.key username.crt username.key /etc/openvpn

Create OpenVPN Config File

Note that previously a version was copied and edited from the /usr/share/doc directory, but latest versions of OpenVPN no longer include this. Instead touch and then use the following file below as the base server.conf:

touch /etc/openvpn/server.conf
cd /etc/openvpn
chmod 0644 dh2048.pem ca.crt server.crt server.key server.conf username.crt username.key

Next, edit server.conf

nano /etc/openvpn/server.conf

Here is an example of server.conf. Ensure the masquerade iptables and server configuration are identical.

port 1194
proto udp
dev tun
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
reneg-sec 0
ca ca.crt
cert server.crt
key server.key  # This file should be kept secret
dh dh2048.pem
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS 84.200.69.80"
push "dhcp-option DNS 84.200.70.40"
duplicate-cn
keepalive 10 120
;cipher BF-CBC        # Blowfish (default)
;cipher AES-128-CBC   # AES
;cipher DES-EDE3-CBC  # Triple-DES
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status openvpn-status.log
verb 3
explicit-exit-notify 0

Enable Routing

nano /etc/sysctl.conf

Change ip forwarding to 1

net.ipv4.ip_forward = 1

Restart networking services

service network restart

Enable and start the OpenVPN service

chkconfig openvpn on
service openvpn restart

Install and Configure OpenVPN Client

For OSX, there is Tunnelblick, which sucks, and Viscosity which sucks less (but costs $9). An example Viscosity config file looks like:

#viscosity startonopen false
#viscosity protocol openvpn
#viscosity dns off
#viscosity usepeerdns false
#viscosity autoreconnect true
#viscosity name host.domain.tld
#viscosity dhcp true
remote 12.34.56.78 1194 udp
nobind
dev tun
redirect-gateway def1
tun-mtu 1500
pull
tls-client
ca ca.crt
cert cert.crt
key key.key
push "redirect-gateway def1"
comp-lzo
mssfix 1450
resolv-retry infinite
tun-mtu-extra 32
reneg-sec 0

For installing an OVPN command line client on Linux, simply taking the config.conf file, along with ca.crt, cert.crt, and key.key files. Installation on a Debian system looks like:

sudo apt-get update
apt-get install openvpn

Then scp the four files into the home directory and run:

openvpn config.conf

After this works, then set up OVPN as a service with scripts for automation. - See also OpenVPN on ChromeOS and Android

Resources Consulted

Posted on Leave a comment

Amazon Customers, Markets, Resources

Amazon is different than other companies. From the outside, this difference can appear as anomolies, odd things that stand out. I belive that there is a fundamental way of understanding the current state and dynamo of change within Amazon.

Customer Centrism

Amazon is meant to be the most customer-centric company on Earth. Likely when we get to Mars, Amazon will need to change that to the Solar System. However, it is not customer-centric in all ways. User interfaces and documentation are a huge challenge that many competitors excel. Even on the issue of price, there are various options that are generally cheaper, or as cheap, such as ebay. This is where things like free shipping and very fast delivery come into play. Bundling digital and physical goods (to sell more physical goods) is a good tactic, as digital goods have low incremental cost of sales. If this locks in a customer who will pay in excess of the margin on digital goods, that is an increase in profit, a happier customer, and an increase in trust and mindshare. But in some cases prices are still poor in relation to the competition. In these cases, it seems clear that (unless we are talking about overlooked anomolies) there are other factors at work.

Resource Centrism

Most organizations have fewer resources than they could use (though that is not correct unless there are appropriate mechanisms for management and leadership). Customer centrism can always be a focus, but of course it will be constrained in terms of resources available. In order to understand customer centrism, there should be two aspects: improvement (which might be called innovation) and listening/understanding (which might be called communication, more generally). Since communication actually informs improvement/innovation, that part is primary (and we see that in the generally lightning-fast reaction times to customer requests. However, this can be somewhat annoying as small things require a request to customer service, rather than being fixed in the interface. However, again, this is brilliant management as those issues which generate the most support calls, or have a greater impact on sales and customer satisfaction, will get the priority for engineering resources. Given enough computing power and good data, prioritization of product development and maintenance could potentially be done by the Amazon computer brain.

Market Centrism

This idea of markets as driving optimal resource allocation is rife throughout the company, and the ability to participate in these markets externally (essentially the core of ecommerce) is invigorating. Some examples:

Amazon Vendor Programs

Amazon has multiple vendor programs, from the original affiliate program and the amazon advantage (for media companies) to the Amazon Seller Central program, there are many ways to interact in the Amazon marketplace. Shipping/delivery is an area that has become increasingly varied with options for vendor fulfillment, fulfillment by Amazon, and in some cases Amazon Prime fulfillment by the vendor.

Amazon Web Services

AWS has itself also evolved over the years, and the fundamental resource of computing, storage, and networking has become sliced and diced into a variety of offerings. EC2 is the basic VPS options with some levels of resiliency. Next came the Elastic Beanstalk, which takes care of much of the provisioning in dealing with scaling stateless services. Third is Lambda which is serverless computing (it simply executes the code without any server management/configuration).

Amazon Hardware - Kindle, Fire

Amazon hardware has also gone through many iterations and includes variety in the offerings. This is akin to a normal product line, and line extensions, though sometimes the level of innovative is category-defining. The Kindle ebook reader, while certainly not the first, has become dominant for good reason. The Fire TV and Fire TV Stick has largely beaten out the Chromecast. And most recently Alexa on the Echo and Echo View.

Software + Hardware + Logistics

The fundamental skills underlying Amazon are several, but building and maintaining software, designing and managing hardware (different kinds, from a handheld device to server farms), and logistics, especially distribution/shipping/delivery. Expect innovation around each of these fairly complex, and difficult alignment of the two sets of three pillars of expertise and practical, valuable knowledge.

Posted on Leave a comment

Thoughts on Amazon Echo Show

The new Amazon Echo Show looks great. Watching the video, it is striking that the interface appears so clean. This is obvious when using voice, since buttons don't really count. Also, the fact that this is not just a piece of hardware and natural language interface, these are applications being shown. In terms of video/voice calls, there is quite a bit of competition, including Facebook, Whatsapp (Facebook again), Viber, Line, Telegram, etc. Oh, and don't forget about Nucleus who seemed to have the idea before Amazon (who is an investor). https://www.youtube.com/watch?v=WQqxCeHhmeU The fact that the Echo dominates the smart speaker market should be a cause for concern by the likes of the slower-moving Google (who bought Nest years ago), not to mention molassas-like Apple has HomeKit, though that requires that OEMs actually buy Apple chips and integrate them into their devices. Amazon's open approach to connected devices is a repetition of Android vs. IOS. Sure, there are a lot of iphones and even ipads, but the notion that this same rather small minority share will be able to co-opt the vast other connected devices, not really thought out very well. The interaction with other Echo Show devices and other smart devices shows off what is at stake here: Echo as a platform for the home.

Missing from Amazon Echo Show, Release 1.0

My first few thoughts about using this new device in the home are what is missing. - There is no video output, but definitely connecting to a monitor or a TV screen makes a lot of sense (that is, it could perform the function of a FireStick). - There is no keyboard input (but it is a touchscreen, of course), perhaps bluetooth mice/keyboards will work? Not a real deal-breaker - Generic browser user included? We did not see anything resembling such. I get it. This is not meant to become a computer. Rather as its' own unique platform, it will be the heart of the connected home. Which leads to the following significant motivation.

The Future Home will require Echo-integrated Devices and Services

Apparently there are an ever increasing number of devices available to Echo. Certain smart hubs can act as extenders, such as the Almond+ which happens to be our Wifi for the first floor. This is an insight for anyone making buying decisions regarding any kind of home device. Even moreso, those software services which Echo can interact with will have an additional layer of functionality based on that fact alone. This means, for example, that my music cannot be stored only in Google Music (where it reposes for free) but I will need to also ship over the 11,000 songs into Amazon Music (for $25/year), and only then will that become available on the Echo platform. That said, there will be ways of doing some DIY integration, such as using Alexa to interact with Foscam cameras via IFTTT: https://www.youtube.com/watch?v=pUwNGSzFvQo

Posted on Leave a comment

Mainstream RHEL Derivatives

Spending time looking into AMI (Amazon Linux), it is as usual with the plethora of Amazon products, sometimes hard to get info about what it is. I take this not as a bad thing (though it does take time) but rather a feature that emerges from the *let's develop lots of stuff all at once" product management genius. The clearest explanation of AMI in relation to other distributions was found on the SaltStack site, as: > Salt should work properly with all mainstream derivatives of Red Hat Enterprise Linux, including CentOS, Scientific Linux, Oracle Linux, and Amazon Linux. This immediately brought to mind the sense that RHEL (and CentOS) are to my knowledge never combined with these other distributions when counting them up. Counting of course does not matter, but it is important when trying to visualize linux for the Enterprise what choices are being made. Ubuntu has a lot of visibility, especially when it comes to configuring and deploying a VPS for small projects. This visibility tends to obscure the latent reality of CentOS, Oracle, Scientific, and increasing in importance, Amazon Linux. Looking at Ansible, the Red Hat deployment tool: > Amazon Linux AMI is mostly compatible with CentOS, but it uses a different version approach, which means that most of those Ansible roles will ignore or complain about not supporting Amazon AMI. One can also use CentOS on Amazon AWS for a more vanilla approach, though Amazon Linux AMI is tuned especially for EC2.

Posted on Leave a comment

Amazon WorkMail, WorkDocs

When dealing with cloud-based office productivity applications (documents and spreadsheets), the main contenders are obvious: - Apple iWork - Google Docs, Apps, Drive, GSuite - Microsoft Office365 While Apple's offering is relatively unknown (little noted, therefore little discussed), the real sleeper is Amazon's WorkDocs and WorkMail

The Kit and the Kaboodle

Here I will deal only with Google and Amazon, rather than all major Cloud providers. I have not considered Apple's (iCloud is an ongoing disappointment) or Microsoft's (sorry, not going back, too many scars). The main issue that I experience with the Google and Amazon offerings is integration among kinds of files to back up, especially images, music, and standard office documents. The main disconnect (for Apple vs. Google) is the Music component. Both are now offering pretty much unlimited images, and a hefty amount of storage. For a small fee ($1.99 USD/month) Google provides 100gb of cloud storage space, plus unlimited images (compressed in some way, not native image size). While the location of Google Photos inside of Google Drive basically works, the edits on one side (filenames, for example) do not replicate to the other side.

Google Mail, Docs, Drive, Photos, Music

For Google, the integration of Mail (and Calendar) with Docs (cloud editors), Drive (all files), Photos (with sync to/from desktop and mobile), all work out well. The Music part is a separate app and sync, but as a basic backup and Web/App based access from other devices, it works great, and offers 50,000 songs of upload storage for free. That's a great backup plus access via any device through a browser or Google Play Music app. I've been using free tiers of Google Apps/Drive for years and years, now grandfathered in. I don't mind paying some small amount per month, I don't need to pay nothing. Currently I pay the $1.99/month for the Google Drive 100gb storage, nothing for Apps, Photos, or Music.

Amazon WorkMail, WorkDocs, Drive, Photos, Music

Amazon has a bunch of similar features with their Amazon Drive and other accompanying products. Drive is unlimited Photos plus 5gb of files for Amazon Prime subscribers, and unlimited files of all types for $59.99/year. They also have a virtual desktop service Amazon Workspaces. For Amazon Music, there are two streaming options (Prime and Unlimited -- which starts at $3.99/mo). But the upload option is the one I am most interested in, and it is 250 songs for free (not including any bought from Amazon, which do not count against the limit), and $24.99/year for 250,000 songs. For pricing, Amazon WorkDocs and WorkMail can be bought together in tiers that support both, for $6 USD/month. Prices are currently $4/month for WorkMail and an additional $2/month for WorkDocs.

Yandex Mail, Yandex Disk

Another interesting competitor not mentioned earlier is Yandex Disk. The latest 2.0 version does not require files to be on a local drive. This solves a problem with most cloud storage, where a thin client can access through a browser, but anything similar to a native client experience synchronizes all files locally. One computer could initially upload a file system, and then later not need those files to remain local, as well as thin clients such as an intel compute stick with limited storage could have a full experience of file interaction. Yandex offers 10gb of space for free, with additional storage purchased monthly at $1 USD/mo/10gb, $2 USD/mo/100gb, and $10 USD/mo/1tb. Yearly purchase offers a 17% discount and any number of each of the tiers can be purchased (example, 1.2tb would be $14 USD/mo or $140 USD/yr. Yandex also has web-based editors for documents, spreadsheets, and presentations. Their sharing model is better than Amazon, as Amazon only allows for comment access, but with Yandex Disk, shared access can be granted.

Mix and Match for 2017

Besides having had generally good experiences regarding Google Drive, Google Apps, Google Mail, and Calendar, these are something I want to migrate off of Google. I simply don't trust them, and there is a lack of encryption involved. With Amazon, WorkMail (and WorkDocs) is available in the US and Ireland currently, so EU laws apply to the Irish servers. As well, the Amazon Simple Email Service (SES) makes it easy to manage transactional email and business use cases. The free Google Play Music is still quite compelling, and I will likely stay on that, as I have close to 100gb (12,000+ songs) in music. Yandex is a great solution for a personal user, there are useful security aspects to it. And while not as configurable at an enterprise-level, it is still a worthy offering. The sync and backup of Yandex Disk appears to be more robust and functional than that of Google Drive. Redundancy for email is difficult to achieve based on the nature of a single email address being mapped to a single mailbox. This still needs to be worked out. However for file backup, filesharing, and editing, using both Amazon and Yandex is a viable solution that is fairly inexpensive (<$10/mo). Note: there is also Proton Mail, which is an even more secure (than Yandex) email-only service.

Posted on Leave a comment

Amazon UCC128 Barcode

Updated 24-Nov-2018

Publishing Toolchain/Workflow Notes

Syntax for ISBN .eps

./bookland -o 978-0-9822972-7-8.eps 978-0-9822972-7-8

Syntax for Amazon ASIN (effectively a hyphen-less ISBN-10 of the ISBN-13)

encode128 0982297270

Take output and use Code128 Font to display

The Amazon Barcode

Note for revision: the latest version of encode128 is using different start/stop characters than what is displayed/discussed below. Need to correct this. The main point is to use encode128 encoder and the code128* fonts from the same developer, which work well together.

Anyone wishing to use Amazon Advantage (for media publishers) or Amazon Fulfillment, needs to include barcodes on individual items, so that they are stored and selected as individual units. Amazon provides the following information in one of their PDFs:

Amazon Barcode Guidelines

If you would like to print barcodes directly on Units, use the UCC128 barcode. The UCC128 barcode standards are available on the Internet.

6.2. Amazon uses the UCC128 barcode (font) to encode the FNSKU or the ASIN in the barcode. We don't use any leading or trailing digits (application identifiers or checksum digits).

6.3. The full specification is UCC128 code set A (this is the code set that supports alphanumeric data).

6.4. If you are building the barcode from scratch, you can review the standards or purchase software (there are many barcode applications available for free or at reasonable prices).

To someone who doesn't know jack about barcodes, this is bewildering and unhelpful. Sure there are many barcode applications available for free or at reasonable prices but where are they, how do they work, and more importantly, which ones do the Amazon thing of UCC128 (which is not actually a name or standard of anything).

And so the adventure begins.

Code 128 A B C

First off, the 128 of Code 128 has to do with ASCII (which is 128 characters), some extended characters (all of Latin-1), and some clever compression (if compressible). Also, the GS1-128 shipping standard (formerly known as EAN) is a subset of Code 128. This means that all shipping and most product identification labels rely on all or some part of how this code works.

Secondly, the A,B,C are slightly different schemes (different character set support) which most system support all of. That is, there is code switching between the different sets, depending on what is encoded. A is A-Z, 0-9; B is A-Z, a-z, 0-9, and some punctuation; C is 00-99. B is the most common scheme (that can be used by itself, but C allows for compression if there is a pattern to the characters (e.g., repeated characters, or characters in series). If nothing fancy is needed then B is fine.

In Code 128 bar code encoding, each character consists of alternating three bars and three spaces (with a possible thickness of 1-4), a start character, a stop character, and just before the stop character is a checksum character, based on a calculation.

Note that each character in a bar code fits into 12 widths (that is, the three spaces and three lines much together add up to 12 widths). In practice, Code 128 is 11 widths, and lines and spaces are 1-4 widths in width. Each character starts with a line, and ends with a space, excluding the final end character which is 13 widths and ends with a 2-width line.

Starting and Stopping Characters

For Code 128 B Character Set, the starting character is Ñ (a capital N with an enya (tilde) overhead).The stopping character is Ó (capital O with an acute). Sometimes these are indicated as a different character, but what is most important, is having a font system that matches the control characters to be used.

Note: don't use the Code128.TTF, instead use one of these better Code 128 Fonts

Calculating the Checksum Character

The basic calculation is a modulo (remainder, after dividing by 103 non-delimiter characters) of the sum of the text string numerical values multiplied by their position. For a text string HELLO, we would see:

  • Start B = 104
  • H = 40
  • e = 69
  • l = 76
  • l = 76
  • o = 79

Formula:

modulo 103 ((104)+(401)+(692)+(763)+(764)+(795)) (Note the 104 for Start B, whereas if the barcode were only a number, it would Start C.) = modulo 103 (40+138+228+304+395) = modulo 103 (1105) = 10 remainder 76 76 is the lowercase letter *L (see Encoded: Hello is: ÑHellolÓ

Practical Note: use the command line code 128 encoder to generate the codes, and then use a Code128* font to render it, or use the bookland python script to generate an .eps file (which is better when needing an encoded ISBN barcde with all the trimmings). Bookland can't just do a barcode, but interprets even an ISBN-10 without hyphens into an ISBN-13.

Bugnote: The code-128 has a problem rendering the character for Capital-I-Diareses (which looks like a divide-by sign as well), so there has to be a fall-back to non-compressed encoding (start and end characters, plus checksum, but no clever encoding). The start and end characters from code-128 are Ò and Ó, and for non-compressed it is Ñ and Ó. A simple LibreOffice Calc implementation of code128 for non-compressed is available.

Amazon Barcode Generation

While in the very first bit of text that Amazon provides, it says it does not use leading or trailing digits or checksum digits. This is not correct (at least from Amazon generated Purchase Orders and Shipping Labels). Amazon uses a Code 128 B scheme (their start character is a B Start) but constrains the codes to A-Z, 0-9 (no lower-case alphabetic characters). They do use a checksum.

Does this really matter? Probably not. Most likely, all user/customer/vendor-generated codes (with or without start/stop/checksum) will be recognized. Implementation for bar code scanners is in software, and even most of the mobile apps that do bar code recognition support codes with and without start, stop and checksums. However, it does make sense to use the same system that Amazon implements. And of course to understand how this works in case of needing to generate missing Amazon barcodes such as Purchase Order or Shipment numbers. And so, now we know.

The simplest approach is two steps:

  • Take the given text string, use Code 128 B and calculate the checksum (as in the example above)
  • Append the start and stop control characters as well as the checksum character
  • Use Code128.TTF font, and then adjust for appropriate width and height (I use Inkscape for this)

Note that this does not use clever compression to produce fewer characters and therefore fewer bars on the barcode.

Note: Better Code 128 Fonts

Bar Code Typeface, Font, and Typography

Because characters always have the same numeric value, and the same bar code encoding, it would be simple to simply type in what will be represented (generally, capital letters and numbers only), and change the typeface, rendering the characters into bar codes. A great, open source, and free TTF font is available from

Online Encoding / Local Scripts

Note, most sites suck over time, and so even if we built a link to something that didn't suck, that would become untrue. Best to stick with code we can manage, that is forked github repositories, namely: bookland python script.

ISBN Tools for Amazon ASINs

Note that Amazon ASINs are not necessarily ISBNs but in many cases they are. When submitting an ISBN-13 as the product code, an ISBN-10 is generated (without hyphens) as the Amazon ASIN.

Note the Library of Congress ISBN Conversion Tool

Online Tools

The Barton site has a handy Barcode encoding tool that generates the start, stop, and checksum characters. This makes it easy to copy/paste into a Barcode Font text and voila, a fully generated (and editable) barcode. However, it uses an old system so the control characters do not match up with the standard found in the Code128.TTF v.2.0. Also, the Barton site has the OLD 1.2 version of the Code128.TTF font. Avoid both.

Encoding and Barcode Generation Implementations

Visual Basic / VBA

Better (though a bit old school) is using Libre Office and a macro as offered by the amazing Grand Zebu, who was also the original source of this open source font (have to enable Macros to run in security settings).

Note: this no longer runs on the latest version of Libre Office.

PHP

Somewhat bloated, half-refactored implementation in PHP (really, a great thing, but monolithic).

Javascript / Node Barcodes

The modern and well-supported tool is the JsBarcode project.

Posted on Leave a comment

Apple Software Updates 20-Sep-2014

In the past three days I've had iOS, AppleTV and OSX updates (though the OSX is not the anticipated Yosemite, rather a 10.9.5 + Safari 7.1 + Xcode 6.0.1 + (10 days ago) iTunes 11.4. I'm actually excited about these updates, whereas with previous Microsoft updates, all was fear. And even with WordPress, too many versions have broken too many things. That said, a previous AppleTV broke Bloomberg for 4 days, so no one is perfect. Looking forward to Yosemite along with my iPhone 5 + iOS 8...