Posted on

Debian on AWS Lightsail

This is a setup of several items, starting with Debian 9 on Amazon AWS Lightsail. This has server basics and apt, and then follows with links to additional articles. In general, after several years of running CentOS on Linode, and then Amazon Linux AMI on EC2 and Lightsail, I find that Debian 9 is simply faster, just as secure, and at least slightly easier to use.

Note: as of Sep 2020, Debian 10 is now available on Lightsail

I will update this soon (mid-2020) to Debian 10 - Bullseye (stable) on AWS and Debian testing on the desktop. I consider this combination to be very good for intermediate users as it keeps them up-to-date on the latest testing build (when things break, that is a learning opportunity), as well as having access to most recent versions of applications, utilities and support libraries. Debian is a huge linux ecosystem which is generally well-supported by a very large community. For one's production desktop environment, Debian testing is an excellent balance of up-to-date application availability and community supportiveness. Together with the extremely stable desktop environment using Openbox/LXDE, very low system requirements are needed.

To be honest, once getting the hang of Openbox/LXDE, I do not see any advantage to Linux Mint or Ubuntu, for that matter (besides the personal repositories). Cinnamon (available on other distributions than Mint) is buggy, memory hungry, and requires a bit of customization. Openbox/LXDE offers nearly the same kind of required customizations, but demands many fewer resources and is nearly crashproof. In my opinion, the good parts of Mint do not include cinnamon, rather applications such as Nemo and Pix, which can of course be installed and run without Mint or Cinnamon.

Continue reading Debian on AWS Lightsail

Posted on

Dropbox Cloud Storage and Sync

Dropbox is a cloud storage and sync service, with additional editors/apps, such as Paper and Showcase. For various reasons, those additional Dropbox apps are not useful for our use cases. However, storage and sync are excellent in and of themselves, and generally superior to Google Drive which is the only real alternative.

What Dropbox gets Wrong

One thing that is maddening about Dropbox is that when renaming a folder, all files and sub-folders within the folder are re-synchronized. This can be a huge undertaking (in terms of time required, not to mention wasted bandwidth. - Dropbox Rule #1: Try not to rename folders Another thing is the slow Microsoft Online Editors for Word and Excel. These can be very tedious to use, and there is more limited functionality than found in native editors for desktop operating systems. - Dropbox Rule #2: Use a native editor on Word and Excel documents when possible. Preview for Word and Excel do not support Indic Scripts (Fonts). This means that any Thai vowel, tone mark, or silent mark will not show properly in preview (but will when editing). This is a very odd limitation, and is based on a very poor preview functionality. In contrast, nearly all other editors support Indic Scripts (South and Southeast Asia-style fonts), with the only other known exception being

Backing up Multiple Folders with Symlinks

Besides what is in the main Dropbox folder (which can be some or all of the contents), there are times when folders in other locations are needed to be included in a backup. To do this, simple create symlinks (symbolic links) from the command line. Aliases created in the Finder do not work as symlinks, so the command line is needed (or some third party app, so unnecessary). The following example points to a folder on an SDCard:

ln -s /Volumes/jm-music/iTunes-Library ~/Desktop/Dropbox/iTunes-Library-symlink
  • ln = link
  • -s = symbolic
  • /Volumes/jm-music/iTunes-Library remote link destination
  • ~/Desktop/Dropbox/iTunes-Library-symlink local link location

Dropbox vs. Google Drive for Mobile Devices

Dropbox is one of the most widely available cloud storage providers in terms of support by third-party mobile apps. While Google Drive has increased its coverage, and Microsoft lags a bit behind, Dropbox is reliably the foremost access provider for cloud storage. As well, the Dropbox app can backup images/video from mobile devices automatically.

Dropbox Desktop Sync Performance

Dropbox is a much better application for synchronization of files, in terms of stability, reliability, and resource utilization (at least on OSX). Google Drive synchronization is a nightmare of processor utilization, hangs, and error messages.

Dropbox Security Audit in Four Steps

Storage in the Cloud does not magically remove the need for security, and especially that rare creature, the security audit. From a post over at Labnol, we learned how to do a Dropbox security audit, which is important for obvious reasons. However, this requires vigilance and a repeated review, something scheduled in your calendar. Note that the user interface at Dropbox changes over time so these steps need to be updated regularly. - Last updated 18 October 2017

Step 1 - Run the Security Checkup

Run the Dropbox security checkup which reviews devices/browser, connected apps, and suggests a password change, as well as review of two-step authentication settings.

Step 2 - Review Devices and Browsers

Check the [devices and browsers which access Dropbox](https://www.dropbox.com/account/security. Anything suspicious?

Step 3 - Review Connected Apps

Review the connected apps enabled to access Dropbox. Anything suspicious?

Step 4 - Review Available Space

Check the Dropbox plan and space used/available. After all, availability is one aspect of security.

Posted on

Open Source Collaborative Docs

We can call this Tsuite, inspired by Toot Sweet (a Chitty Chitty Bang Bang candy invention), and is meant to provide some functionality offered through third party collaborative documents. The main point is to have a self-hosted, free-and-open-source alternative, albeit with more bare bones functionality. Ultimately the goal is to be functional enough to allow for the replacement of things like Google GSuite, Microsoft Office Suite, and the like. Note: This topic is continued with a recent article about Dropbox Paper

Basic Functionality

  • Email (with IMAP)
  • Web-based Spreadsheets
  • Web-based Text documents and/or wiki pages
  • Some level of collaboration (realtime synchronous and serial persistence)
  • Some level of security and access control (though rudimentarily may be VPN-based)

Expanded Functionality

  • User and group access control
  • Better audit control and backup
  • Export of documents into common document types (.pdf, .xls, .doc)
  • Import of documents
  • More extensive wiki layouts
  • Shared calendaring
  • Shared contacts

Tools to Start

File Options to Explore

Notes

For a third-party, open source solution, the most complete while having a modicum of stability, is likely Pydio + Collabora on GlusterFS. Pydio has mobile apps and integrates with Collabora, which has web-based file editors, and GlusterFS is the file service on the server. It is possible that the Pydio open source docker image would be enough, including (or in addition to) the Collabora Online Development Edition distribution. The biggest problem with Collabora as a shared document editing environment is the requirements for a server, which is 8gb of ram (currently $76/mo for an AWS EC2, though a 1-year reserved instance would provide a 38% savings at $46.57/mo). Note: This topic is continued with a recent article about Dropbox Paper

Posted on

Dropbox Paper, Markdown, Sync

Dropbox Paper is a product I really want to like. For one thing, the promise of better editor is something long unfulfilled. And taking some design cues (or perhaps merely unrelated similarities), Medium did do something nice for the blogging environment. By extending it as essentially a wysiwyg Markdown+ editor, drag and drop-friendly, with handy JavaScript handles for visual editing, this is definitely an interesting project.

Markdown as a First Class Filetype

However, there are those of us who prefer something with Markdown as a first class document filetype, which could be seamlessly synchronized alongside other files, and edited with other editors. This is what the cloud editors do, after all, provide some level of editing of desktop-class documents, collaboratively, and those same files are generally available in the same binary format (via sync or import/export). Of course by Markdown we mean much more than the anemic initial (but no less necessary) initial Markdown spec. We like Markdown Extra as a more complete specification. With Dropbox Paper there is at least one additional feature from straightforward Markdown, the inclusion of images without having to know where they physically reside. Drag in and the image appears as a part of the document. This is similar to what Github does well with its Github-flavored Markdown. Clearly there is some kind of zip/archive file format behind the scenes, which we simply don't have access to, or perhaps a nasty rats nest of pointers in a database. The thing is, Paper doesn't have a public spec or source available. In other words, Paper documents only live in Paper, the Application (akin to Google Docs). This means Paper is not a first class filetype, and therein lies the rub.

Stable File Format is Key to Offline Sync, Editor Diversity

With its Office-in-the-Cloud, Microsoft actually preserves both the file format and allows a variety of editors (which is what preserving the file format enables). This in a superior way with Word and Excel documents, essentially round-tripping edits into synced files in the desktop or the cloud. Libre Office Online, Collabora CODE, and Collabora Online are similar to the Microsoft approach, with files not changing their basic structure. Of course one would expect Microsoft to take this approach of focus on file format, since it is what helped cement their leadership in editing applications. Own the format, own the tools. Google took a different approach (for scalability reasons, surely), and the ability to edit anywhere requires offline applications and the use of a browser (which means lock-in to the Google Suite editors). Dropbox Paper is less functional than Microsoft, Libre, and Google suites, but appears to be taking a Google approach, sad.

Scalability and the Requirements for State Management

A quick read of the EtherCalc story provide excellent insight into what it takes to maintain state in a connection-less multi-user environment. Essentially a copy of the document needs to be kept updated. That generally requires the same resources as on a workstation, and was likely a strong motivation for Google building from scratch simpler non-compatible file formats for Word and Excel documents, as well as a log of all change-sets (for version tracking). The same is said for the Pydio/LibreOffice Cloud offerings, namely that they take a bit of memory to get them to work, again due to the architectural requirements of real-time server-based state management.

Limitations of Paper as a First Class Editor

Since Dropbox already uses the Microsoft Online editors for Word and Excel documents (which work as advertised), Paper is a bit of an unwanted stepchild in terms of integration. Paper (which is both a file storage and a file editor, with web and mobile app versions) doesn't live within the Dropbox folder system, but rather has its own file system. This is awkward, for navigation, to say the least. Paper folders and files do not sync.

Offline Editing with Mobile, but not Web

With the IOS or Android App, Paper files can be edited offline, akin to Google Suite, but without the ability to do offline editing with the web app. Paper files can be exported in Markdown and Word document formats, but there is no ability to import Paper files, one has to copy/paste. This is trouble if someone has a lot of Markdown files already at hand. It seems clear that this use case is fairly well ignored.

Copy/Paste and AutoCorrect in Dropbox Paper

With copy/paste, another glaring problem comes to light, which is the required transformation of single and double quotes into their fancy quotes equivalents. This is a non-starter for people working with text that needs to remain inviolable in their originally intended ASCII characters. There are obvious work-arounds for the well-known ASCII and Unicode Quotation Marks problem, and a handy visual JavaScript replacement is one that would keep the underlying text unchanged, but that is not the path which Dropbox took on this editor. Rather like Microsoft Word and Google Docs -- but unlike them without the ability to turn off the AutoCorrect options -- characters are automatically replaced, not leaving a trace of what they were formerly.

Mo' Paper, Mo' Problems

Intimated above is that what went wrong is a lack of open source of the file format that Dropbox Paper uses, which gives it severe technical limitations in terms of portability, offline editing, file synchronization, and a clear separation between interface and file format. This approach is one which Google has also embraced, and Google's moderate success in the face of such technical limitations should not be a signal of the weakness of such limitations. Rather one should look at the years and years of massive resources poured into the cloud editing project, which still cannot do proper offline file synchronization, and which has allowed Microsoft to compete effectively after a very long delay in entering the market with cloud editors.

Conclusion: Ignore Dropbox Paper

For the particular use case mentioned above, requiring file format and content integrity, file synchronization, diversity of editors, and the like, the solution is to simply ignore Dropbox Paper. The product is not for me or others with my same requirements. Fair enough. For our needs: - Dropbox for file synchronization - Stackedit version 5 for web-based editing (with access to the Dropbox file system) - Editorial for IOS is a great plain text editor supporting Markdown and Fountain (screenplay formatting), and also includes workflow scripting with python. There's a book on doing workflow with Editorial. - Atom editor Updated needs (18-Sep-2018): - Google Drive (Gsuite) + Insync for Linux - for files only, no online editors - Text - Chrome Extension, Excellent - Caret - Chrome Extension, Also Excellent - Atom editor - Native Application, Cross-Platform - Libre Office - Clunky but Functional

Posted on

Niche Search

There are several white label options for search and if one considers advertising a viable business model to engage in, then consider Bing or Infospacehttp://www.infospace.com/partners/. Consider the case of Izito and related country code tlds, as well as MonsterSpace, both are pure play partner deals with nothing technical in their own right, with traffic and profitability. The best numbers come out of www.ecosia.org, which publishes their financial reports. I guess if the alternative is a search engine that doesn't provide some kind of additional value, that would be good. For sheer scale it is important to do a general purpose search engine. And then also having some level of curated content overlaid that would be black-hat proof, as well as some kind of vetting of ads?

Posted on

PHP and MariaDB on Debian

Note: instructions for installing and configuring phpMyAdmin also included below.


Related Artices in Debian Services and Applications - Debian on AWS Lightsail - OpenVPN on Debian + UFW Firewall - Nginx and Letsencrypt on Debian - PHP & MariaDB on Debian - Grav CMS on Debian


As of December, 2018 there are decent performance gains with the latest PHP and MySQL (MariaDB, not Oracle) versions. These are: - PHP 7.3.0 released 06 Dec 2018 - Next PHP release 7.4 likely out December 2019 - MariaDB 10.3.11 released 20 Nov 2018 - Latest MariaDB release 10.4 is in release candidate status as of May, 2019. It would be good to do a new version along with PHP when it's next is released, say Dec 2019/Jan 2020.

PHP 7.3 outperforms PHP 7.2 and earlier versions on nearly all real-world web cms platforms. At the same time, MariaDB does indeed have performance enhancements which generally make it faster than the Oracle offering. For MariaDB the performance advantages have been apparent since at least MariaDB 10.1 vs. MySQL 5.7 back in 2014.

This is no surprise, being that MariaDB was founded and developed under the direction of the original MySQL founder. The main advantages technically are better thread management and defragmentation of the MariaDB than MySQL databases. In addition, a larger variety of engines are available under MariaDB including NoSQL (Cassandra).

Set up PHP Repository and Certs

sudo apt-get install apt-transport-https lsb-release ca-certificates
sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list

Update and Install PHP

Currently this is the 7.3 branch

sudo apt-get update -y
sudo apt-get install -y php7.3
sudo apt-get install -y php7.3-cli php7.3-common php7.3-curl php7.3-fpm php7.3-gd php7.3-json php7.3-mbstring php7.3-opcache php7.3-readline php7.3-xml php7.3-intl php7.3-zip
php7.3-mysql

Update and Upgrade apt

sudo apt update -y
sudo apt upgrade -y

Verify php-fpm status

systemctl status php7.3-fpm.service

stop injected data into server returns

sed -i 's/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/g' /etc/php/7.3/fpm/php.ini
systemctl restart php7.3-fpm.service

Edit php7.3 php-fpm conf file if needed, e.g., increase upload size variables.

nano /etc/php/7.3/fpm/php-fpm.conf

Make the following changes:

cgi.fix_pathinfo = 0
...
max_execution_time = 300
...
upload_max_filesize = 32M
...
post_max_size = 32M

MariaDB - Install cert manager, key, repository

currently 10.3

sudo apt-get install -y software-properties-common dirmngr
sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
sudo add-apt-repository 'deb [arch=amd64,i386,ppc64el] http://mirrors.dotsrc.org/mariadb/repo/10.3/debian stretch main'

Then perform update and install mariadb-server

sudo apt update -y
sudo apt-get install -y mariadb-server
sudo systemctl status mariadb

Enable auth socket

sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

Add plugin-load-add = auth_socket.so in the [mysqld] section. Then save and restart MariaDB.

sudo systemctl restart mariadb.service

Secure the database

sudo mysql_secure_installation

PhpMyAdmin on Debian

Provided that Nginx and LetsEncrypt SSL is installed and configured. It is time to install PhpMyAdmin

sudo apt-get update
sudo apt-get install -y phpmyadmin

Add a symlink from /usr/share/phpmyadmin to /var/www/html or whatever directory for whichever website

sudo ln -s /usr/share/phpmyadmin /var/www/html

Note for security through obscurity, rename the link

sudo mv /var/www/html/phpmyadmin pma

Install and enamble mcrypt in php, and restart php-fpm

sudo apt-get install -y mcrypt
sudo phpenmod mcrypt
sudo systemctl restart php7.3-fpm

Test to see if it works

https://host.domain.tld/pma/

Limit access to /pma/ by ip address, by editing the nginx configuration

nano /etc/nginx/sites-available/default

Add the following line to the top above server:

geo $admin { default 0; 203.150.176.16 1; }

And put a nested statement under \.php as per this StackOverflow answer

location ~ \.php$ {
    location ~ (/phpmyadmin/) {          # add this
        if ($admin = 0) { return 404; }  # add this
        ## fastcgi parameters            # duplicate these lines
    }                                    # add this
    ## fastcgi parameters ##
}
Posted on

Google Apps Free vs. G Suite

For those of us early adopters who managed to snag a 50 user (or 10 user) free google apps accounts, things are sweet. New users have to cough up $ 5 USD/mailbox/month (or $ 4.19 USD/mailbox/month provided yearly billing). What Google really wants is the $ 10 USD/month or $25 USD/month enhanced cloud services. Yeah, right. Ok, but what if you need more, or different, domains configured for the free account? Google has systematically cracked down on the ability to add secondary domains (only domain aliases allowed beginning a few years ago), and now it is not possible to change the primary domain either with a free account, or by temporarily signing up for a trial account. The handy ability to add full secondary domains within a trial account means it is not possible to downgrade. However, all is not lost. There is one final loophole that exists (currently) that is still useful.

Secondary Domains vs. Domain Aliases

The main issue is one of namespace. A domain alias (almost) always means all users with a given account (e.g., info@, name@, etc.) have full alias across all domain aliases added. This is not useful if one wants separate mailboxes for separate domains, with the same username (e.g., info@domain1.com, info@domain2.com).

Mailboxes, Email Aliases, Mailbox Forwarding

What should be discussed ahead of this is how to manage mailboxes. For a single person, a vast proliferation of mailboxes is not all that great, either now or in the future. Mail clients themselves can handle multiple mailboxes, but also mailboxes can have mail aliases, and that is possibly all the functionality actually required. In addition, mailboxes can simply forward email as well. The advantages are: instead of a regular mailbox, it does not need to be logged-into, and mail is stored in a primary mailbox (filters/rules can help with tagging/sorting mail). The advantage over mail aliases is that the mailbox can be logged into separately, and therefore managed separately. This is handy when an info@ mailbox is managed by someone other than the sysadmin, who has the admin@ mailbox as their basic utility mailbox for things such as social media accounts, advertising accounts, website mailer accounts, etc. In sum, there are three basic configurations: - Mailbox, logged-into with a mail client (for monitoring) - Mailbox, forwaded email (not logged-into, normally) - Mailbox alias, cannot be logged-into (without primary mailbox access) Let's walk through the main use scenarios to see what is actually needed for certain situations, a unique mailbox (with or without forwarding) or a mail alias.

Postman SMTP - Secure OAuth2 Gmail from WordPress

A very useful and secure email client, Postman SMTP is a great option. The idea is that a single correct email address needs to be used for sending and replies. With two domains and two WordPress sites, is it possible to have info@domain1.com and info@domain2.com as the primary sending email addresses while using a single mailbox? No, not really. Any given mailbox requires a primary email address that acts as the username (at least with Google/Gmail/Gapps/Gsuite). For simplicity's sake, the domain name cannot be different from the WordPress website, nor can the email address be different from the send-as email address of the mailer (reply-to can be different, however). Having a dedicated account for this kind of utility, including troubleshooting as needed, makes work much easier, than having to ask access, and have that access granted, then revoked, each time it is needed.

Paypal Email Account

It is possible to transact with one paypal email address (the underlying primary email address) and use an alias email address. This second email address is the one visible on transactions and that receives email from Paypal. From both the Postman SMTP and the Paypal examples, it is desirable to have a branded info@domain.tld, and it is necessary to have multiple accounts, at least 1 per domain. In addition, for security it is best to have a dedicated account with administrative rights to email (to be used only for admin purposes, and not as a general purpose mailbox).

Free Google Apps and Unique Non-Aliased Mailboxes

There is one way of creating mailboxes that can maintain unique accounts on a per domain basis. This is basically a configuration work-around. The process is as follows: - There will be one mailbox with the standard info@primary-domain.tld. This will need to be configured in a number of ways. - For each domain where an info@ is desired, create a special mailbox with the account domain@primary-domain.tld - Once these are created, log into each of them to perform basic configuration, as well as add the info@domain.tld as a secondary account (validate with the info@primary-domain.tld), and set this as primary. - At the gmail account of the info@primary-domain.tld. There are three things to do: - Configure the mailbox as having other accounts (each of the info@domain.tld, one for each domain alias created) - Create filters (which act as mail forwarding), for each of the domains, such as any email coming to the domain-alias.tld will be forwarded to domain@primary-domain.tld - Validate the email addresses that are used in the filters (replying with links or the security code to enable the filters - Once this is done, there is a mailbox that can receive email sent to info@domain.tld (forwarded from the info@primary-domain.tld), and can send email as well from that address. Most mail clients can that simply add each mailbox in turn, without having to have aliases for the individual mailbox (not always supported in mail clients). The key is to log into the mailbox with the domain@primary-domain.tld account, but change the sending information to info@domain.tld. A small price to pay for ongoing free access to Google Apps for Domains.

GSuite API - Cute, Not Useful

I'm all for APIs but instead of a developer demonstrator, it would be way better to have a real tool, backed by an API (or not, who cares?). Here are a few clumsy things to do: - Add a new secondary domain (doesn't work on free or trial accounts) - Get a list of domains on the account - Set the primary domain (doesn't work on free or trial accounts) - Get customer info, including immutable customer ID (CustomerKey is my_customer, also need to authenticate)

Additional Notes

If moving a domain from one Apps/Gsuite account to another (and in the process, deleting the first account), there is about 24 hours where it will be unavailable to register/validate in the receiving account. Prepare for this by changing MX records and forwarding any email addresses that would be impacted. When deleting a Google Apps account (and any mailboxes and accounts within it, it is important to have granted ownership to a variety of possible additional accounts linked to it, including: - YouTube channels - Google+ Accounts - Google My Business - Locations - Brand accounts - Google Adwords - Google Adsense - Google Analytics The list goes on, so think it through.

Posted on

Amazon Customers, Markets, Resources

Amazon is different than other companies. From the outside, this difference can appear as anomolies, odd things that stand out. I belive that there is a fundamental way of understanding the current state and dynamo of change within Amazon.

Customer Centrism

Amazon is meant to be the most customer-centric company on Earth. Likely when we get to Mars, Amazon will need to change that to the Solar System. However, it is not customer-centric in all ways. User interfaces and documentation are a huge challenge that many competitors excel. Even on the issue of price, there are various options that are generally cheaper, or as cheap, such as ebay. This is where things like free shipping and very fast delivery come into play. Bundling digital and physical goods (to sell more physical goods) is a good tactic, as digital goods have low incremental cost of sales. If this locks in a customer who will pay in excess of the margin on digital goods, that is an increase in profit, a happier customer, and an increase in trust and mindshare. But in some cases prices are still poor in relation to the competition. In these cases, it seems clear that (unless we are talking about overlooked anomolies) there are other factors at work.

Resource Centrism

Most organizations have fewer resources than they could use (though that is not correct unless there are appropriate mechanisms for management and leadership). Customer centrism can always be a focus, but of course it will be constrained in terms of resources available. In order to understand customer centrism, there should be two aspects: improvement (which might be called innovation) and listening/understanding (which might be called communication, more generally). Since communication actually informs improvement/innovation, that part is primary (and we see that in the generally lightning-fast reaction times to customer requests. However, this can be somewhat annoying as small things require a request to customer service, rather than being fixed in the interface. However, again, this is brilliant management as those issues which generate the most support calls, or have a greater impact on sales and customer satisfaction, will get the priority for engineering resources. Given enough computing power and good data, prioritization of product development and maintenance could potentially be done by the Amazon computer brain.

Market Centrism

This idea of markets as driving optimal resource allocation is rife throughout the company, and the ability to participate in these markets externally (essentially the core of ecommerce) is invigorating. Some examples:

Amazon Vendor Programs

Amazon has multiple vendor programs, from the original affiliate program and the amazon advantage (for media companies) to the Amazon Seller Central program, there are many ways to interact in the Amazon marketplace. Shipping/delivery is an area that has become increasingly varied with options for vendor fulfillment, fulfillment by Amazon, and in some cases Amazon Prime fulfillment by the vendor.

Amazon Web Services

AWS has itself also evolved over the years, and the fundamental resource of computing, storage, and networking has become sliced and diced into a variety of offerings. EC2 is the basic VPS options with some levels of resiliency. Next came the Elastic Beanstalk, which takes care of much of the provisioning in dealing with scaling stateless services. Third is Lambda which is serverless computing (it simply executes the code without any server management/configuration).

Amazon Hardware - Kindle, Fire

Amazon hardware has also gone through many iterations and includes variety in the offerings. This is akin to a normal product line, and line extensions, though sometimes the level of innovative is category-defining. The Kindle ebook reader, while certainly not the first, has become dominant for good reason. The Fire TV and Fire TV Stick has largely beaten out the Chromecast. And most recently Alexa on the Echo and Echo View.

Software + Hardware + Logistics

The fundamental skills underlying Amazon are several, but building and maintaining software, designing and managing hardware (different kinds, from a handheld device to server farms), and logistics, especially distribution/shipping/delivery. Expect innovation around each of these fairly complex, and difficult alignment of the two sets of three pillars of expertise and practical, valuable knowledge.

Posted on

Xiaomi – Brand on the Rise

In my household we have been introducing Xiaomi products for the past six months. Overwhelmingly positive, but not without a hiccup here and there. In general, there is an odd mixture of: quality, design, and value. I say odd because generally those things don't go together. Great price, good looks, and works well. Not perfect, but nothing is. Xiaomi may have much bigger rivals, especially in China, but they have such a strong combination that they are able to compete with, and in some cases beat out such giants as Samsung in markets like India.

Continue reading Xiaomi – Brand on the Rise

Posted on

Tokyo or Singapore Datacenter

Tokyo still top choice for Asia HQ Datacenter

Even with multiple options coming online in Singapore, Tokyo is still the best option for a single, Asia-based datacenter. Even when serving locations such as Thailand and Vietnam, Tokyo is a strong competitor to Singapore. Linode is my unmanaged VPS of choice. They have a great service, are as reliable as any other, generally very speedy responsiveness, and great prices. They also have a number of locations, including: Germany, Japan, Singapore, the United Kingdom, and the United States.

When Singapore is a Better Datacenter Choice

  • UPDATE 21-Nov-2016: Tokyo 2 is now publicly available at Linode, my testing has continued to back up the assertion that Tokyo is still the best spot for an Asian HQ Internet presence. My current Tokyo datacenter can't accept new customers, or do any hardware upgrades as it is completely sold out (and has been for a few years now). However, a new Tokyo datacenter will come online by the end of 2016, which is fairly easy to migrate to. Currently the best option is a Singapore location, then moving to Tokyo when the new system comes online. Or just wait until Tokyo is available and make the move then. Those are the two options. The only time Singapore would be a better choice, would be if Singapore, and/or Malaysia, and/or Australia are the primary markets, or if there is some kind of legal requirement for being in Singapore.

For East Asia, Little Difference between Tokyo and Singapore

To Australia, a Singapore location would be slightly faster, and to the US and UK, slightly slower. Submarine lines (that run most of the backbone Internet traffic, as it doesn't go through other countries, more secure). More fiber is being built between Singapore and Australia (some of it came online in 2015). There is a significant interconnect between Tokyo and Singapore (and other countries in between) coming online this year. Therefore to locations such as Malaysia, Thailand, Vietnam, Hong Kong, and Taiwan there is little difference.

Routes to Australia - Tokyo Leads the Pack

This section was added in August, 2017 After six months of suffering with a Singapore location, I've finally come to the conclusion that raw pipes are definitely not enough. Others have made the same argument. To summarize from not only ping speeds but better, fully-loaded (but optimized for speed) ecommerce sites when trying to reach the Australia market: - Tokyo is the fastest connect to Sydney (besides Sydney itself), however there is some delay to Europe - Sydney itself is superfast to Australia but sucks to everywhere else. - Northern California is very fast to North America, and acceptable to Sydney and to Europe - My estimation is as follows: - About 1 second load time on an optimized WooCommerce site with caching - Another 0.5 second for transit within a fast region (NA, EU) - From N.Calif, add 1.5 seconds to EU and 1.5 seconds to Sydney (and 0.5 seconds within NA) - From Tokyo, add 1 second to Sydney, 1 second to North America, and 2 seconds to the EU - From Singapore, add 1.5 seconds to Sydney, 2.5 seconds to North America, and 3.5 seconds to the EU - From Sydney, add 3 seconds to the EU and 2 seconds to North America, and 0.5 seconds within Australia - Tokyo is a balance of fast to Sydney and North America, and moderate speed to Europe - Northern California is a balance of moderate speed to EU and Sydney, and fast to North America - Sydney is a balance of fast to Australia, and slow to EU and North America Note that any edge location can be sped up with a content delivery network (one that works). P.S., AWS Tokyo also has Lightsail which is Amazon's VPS bundle, that includes things like a decent amount of data transfer, disk, 3 dns zones in Route 53, etc., at no extra charge.

Arctic Fiber Lines

A few arctic fiber lines being deployed which will cut time from Tokyo to the US West Coast as well as Europe.

Single Location - Tokyo

In summary, Tokyo still appears to be the best single place in Asia to be located (which is why it is so popular, and generally more expensive). Tokyo has decent speeds to Europe, more redundancy, and faster speeds to the US (already a fast link), and to Australia (via Sydney).