Load Many Images in a Single Request (Optimization)
Some pages on GoToQuiz.com generate many requests to retrieve many thumbnail-sized images. Each image file is only 1 – 5 KB in size. I wanted to reduce the penalty associated with making numerous requests, so I came up with a way to combine these into a single request.
I needed a solution to retrieve an arbitrary number of images, which may vary with each pageview. I considered generating an image sprite on the fly, but this approach came with several problems:
- Constantly invoking image functions on the server, adding more overhead
- Image display would require using background-image and CSS manipulation
- Image display would also depend on meta data about each image, to correctly set the necessary CSS values
- Images may be different heights and widths, further complicating sprite generation
So instead, I went with returning the image data base64-encoded in a JSON format. It works like this. The HTML will have elements each holding the URI of the image as a data element rather than in <img src=”…”>.
<div class="gImg imgSwapHiPri" data-imgpath="/upic/2017/21/Mn8rrw.T1.jpg"></div>
In this case, it’s a certain kind of thumbnail styled with .gImg to have a specific height, width, and generic default background. The .imgSwapHiPri class is used to tag all elements to have their images swapped in.
And here’s the jquery plugin I wrote, to be called on pageload:
(function($, dataAttr) { $.lazyImage = function($swaps, alt) { $swaps = $($swaps); var thumbs = $swaps.map(function() { return $(this).data(dataAttr); }).get(); if (thumbs.length !== 0) { $.get('/img/image-set.json', {"paths[]": thumbs}, function(json) { $swaps.each(function(i, el) { if (el.tagName === 'IMG') el.src = json.base64[i]; else el.innerHTML = '<img src="'+ json.base64[i] +'" alt="'+ (el.getAttribute('data-alt') || alt || '') +'">'; }); }); } }; })(jQuery, 'imgpath');
Very simple. Invoked with $.lazyImage(‘.imgSwapHiPri’);. You can pass an optional alt attribute value, or optionally set a data-alt attribute per element. This script checks if the element to be swapped is an IMG tag; if so, set it’s src attribute. If not, set its innerHTML to an IMG tag. Either way, the image data is set to be the base64-encoded image returned by /img/image-set.json.
Let’s take a look at what’s happening in /img/image-set.json’s PHP code. I’ve got some framework boilerplate in the actual script, but the main bit is this:
const WEB_ROOT = '/path/to/public_html/'; $paths = $_POST['paths']; $b64 = array(); //fill an array with the base64-encoded images foreach ($paths AS $path) $b64[] = base64EncodeImg($path); header('Content-type: application/json'); echo '{"base64":'; echo json_encode($b64); echo '}'; //function to convert image path to base64 data function base64EncodeImg($path) { //first ensure the image path contains only legal characters if (preg_match('@^([-_/a-z0-9]+\.)+(png|jpg|gif|svg)$@i', $path)) { //absolute path to file $path = WEB_ROOT . (strpos($path, '/') === 0 ? substr($path, 1) : $path); if (file_exists($path)) { $type = pathinfo($path, PATHINFO_EXTENSION); $data = file_get_contents($path); return 'data:image/' . strtolower($type) . ';base64,' . base64_encode($data); } else return ''; } else return ''; }
The data generated by the code above is a simple JSON file containing an array of base64-encoded images. Critically, it is generated in the same order as the elements in the HTML. This allows the jquery plugin to easily match base64 data with the appropriate element just by iterating over them.
And that’s really all there is to it. When the pageload event fires, the jquery plugin is called and makes the request for, say, a dozen images. The image data is returned in one JSON response–typically under 30KB in my case with JSON gzipping turned on (mod_deflate)–and it is fast.
BONUS:
If you have alternate high resolution versions of images for high DPI screens, this modified version of the above jquery plugin has your back.
(function($, dataAttr) { $.lazyImage = function($swaps, alt) { $swaps = $($swaps); var hdpi = window.devicePixelRatio || 1 > 1.3, thumbs = $swaps.map(function() { var $this = $(this); return hdpi ? $this.data(dataAttr + 'H') || $this.data(dataAttr) : $this.data(dataAttr); }).get(); if (thumbs.length !== 0) { $.get('/img/image-set.json', {"paths[]": thumbs}, function(json) { $swaps.each(function(i, el) { if (el.tagName === 'IMG') el.src = json.base64[i]; else el.innerHTML = '<img src="'+ json.base64[i] +'" alt="'+ (el.getAttribute('data-alt') || alt || '') +'">'; }); }); } }; })(jQuery, 'imgpath');
In this case, the plugin checks if the screen is high DPI and checks each element to see if there is a data attribute called “data-imgpath-h”. If so, it uses this instead of the value of “data-imgpath”. The value of data-imgpath-h is the path to the high resolution version of the image.
A Super-Lightweight jQuery Toast Plugin
Looking for a lightweight toast jQuery plugin? Here’s a super simple one I wrote for this site, called gtqToast. It minifies to just 520 bytes.
(function($) { 'use strict'; var $toastList = $('<ul id="gtqToast"></ul>').appendTo('body'); //register the plugin $.fn.gtqToast = function(options) { var settings = $.extend({}, {fade: 500, linger: 4000}, options), $toast = $('<li style="position:absolute;visibility:hidden;"></li>') .appendTo($toastList), $container = $('<div></div>') .html(this) .appendTo($toast), height = $toast.height(); $container.hide(); $toast.css({display: 'none', position: 'static', visibility: 'visible', height: height}) .slideDown(160, function() { $container.fadeIn(settings.fade) .delay(settings.linger) .fadeOut(settings.fade, function() { $toast.remove(); }); }); }; $.gtqToast = function(text, options) { $('<span></span>').text(text).gtqToast(options); }; })(jQuery);
It needs this bit of CSS as well (note that I assume you use a CSS reset, otherwise you’ll want to add properties like list-style: none
):
#gtqToast { position: fixed; bottom: 10%; z-index: 102; text-align: center; width: 100%; } #gtqToast div { display: inline-block; background: rgba(0,0,0,0.75); color: #fff; font-size: 15px; font-weight: bold; padding: 14px 20px; border-radius: 20px; box-shadow: 3px 3px 9px 1px rgba(0,0,0,0.3); margin-top: 8px; }
What the plugin does: inserts an empty ul
(unordered list) tag into the DOM with a fixed bottom position 10% up from the bottom of the viewport. This list will hold the toasts. You can trigger a toast in two ways.
// #1, call on a jQuery object $('<span>This is my toast</span>').gtqToast(); // #2 call with text $.gtqToast('Another toast, yay!');
These calls will add the toast to our list and fade them in. If you look at the plugin code, I actually add a div
inside the li
to contain the toast message. Plus you’ll notice some weird CSS changing. I did this to account for multiple toasts becoming visible at the same time. What will happen is, existing toasts will slide upward when a new toast is added, then the new toast will fade in.
There are only two settings you can change on this basic plugin: fade is the fade in/out speed, and linger is the length of time the toast will stay on the screen. The defaults should be fine for most cases.
Feel free to use this if it fits your needs!
New Project Launched: Spectacular.Gift
It began like this: Amazon.com’s vast product catalog contains many clever and unique items, the sort that you may not know you wanted until you’ve heard of it. Alternately, these items might make an ideal gift when shopping for the person “who already has everything”. So I figured it would be a neat idea to curate a collection of these items and build a gift recommendation site around them. Doing so would allow me to explore some new server-side technologies and help keep my skills fresh.
Technologies used:
- Ubuntu VPS from Linode
- Apache2 HTTP server
- Apache Tomcat 7
- Apache ActiveMQ
- Spring 3.1 with Spring MVC
- PostgreSQL 9.3
- HTML5 + CSS3
- jQuery, flot, TinyMCE
- Java libraries such as Joda Time, Jsoup, Jackson, Logback, and Commons DBCP
- Amazon Product Advertising API
- Reddit REST API
To get some data populated in the database as a starting point, I set up a scheduled task to pull data from several Reddit forums where Amazon links are shared. Reddit conveniently makes this data available via their REST API. All products discovered in this way are set to unapproved status pending manual review.
Next, I set up another scheduled task to populate and refresh metadata about the products via Amazon’s Product Advertising API. Per Amazon’s terms in order to display price information, this data has to be refreshed hourly. For efficiency I request data on batches of ten products at a time, which is the maximum limit.
I created a manual process for reviewing and approving products to be shown. This process includes writing a custom description, adding relevant tags (e.g. “For Kids” or “Gadget Lovers”), and setting an age range and gender specificity (if applicable).
The UI is written in JSP and outputs HTML5. Some features are powered by javascript, such as the price history button which uses flot to render the graph of historical price data.
Spring 3.1 ties it all together. Spring MVC handles the front-end. Spring JDBC is used for interacting with PostgreSQL. I could have used Spring’s event system, but I wanted to get some experience with ActiveMQ. There are a number of message senders and listeners set up for events such as “price changed” or “product suggested”.
I’ll probably think of a snappier name eventually, but for now I registered http://spectacular.gift (new “.gift” TLD). Have a look if you like! It’s basically in beta, and I’m still adding new products and tags.
Easy CSS Ribbons / Banners (with an assist from SVG)
Ribbons, banners, whatever you’d call them. Put some text inside a ribbon using only markup like this:
<span class="ribbon">1st Place</span> <span class="ribbon silv">2nd Place</span> <span class="ribbon brnz">3rd Place</span>
And it should look like this:
So I created a fairly simple ribbon vector image using Adobe Illustrator, saved to SVG format, and hand-edited it to this:
Recreating a Logo in SVG (from a PNG)
This is the hdpi logo for GoToQuiz.com, in PNG format:
It’s file size is only 3.83KB, which isn’t bad at all. And it looks crisp on hdpi screens. But I wanted to try recreating it in SVG anyway, mainly as a learning exercise.
Getting the basic outline of the letters was easy, given how blocky they are. I had Adobe Illustrator trace them and output the result as an SVG. This would be my starting point for hand-coding the rest.
First, I rounded off each point that Illustrator generated to integers. For such simple shapes, integers really just make it easier to ensure everything is lining up. A few lines ended up a pixel off from where I wanted them, but that was easy to correct. So this was my starting point:
Web Page Optimization – Changes I Made for Page Loading Speed
It has been years since I optimized the page loading speed at GoToQuiz. At that time, image sprites and CSS/javascript minification were the state of the art. In the intervening years, though, further progress has been made. Of particular note is the “retirement” of older versions of Internet Explorer, and the emergence of CSS3 and HTML5, which allows developers the ability to further streamline their sites. So as I am in the process of revamping the UI, I’m taking advantage of the latest techniques to boost page loading speed. Be aware, this is nothing revolutionary–this blog post is simply an overview of which changes I’m making, and why.
Migrating pages over to HTML5 allows the markup to be more semantic while taking fewer bytes–a double win. Using tags such as nav, section and article helps alleviate so-called “div-itis”, allowing cleaner markup and a smaller overall filesize. HTML5 also follows the maxim of “convention over configuration”, letting you eliminate unnecessary attributes like type=”text/javascript” on script tags, for example.
The adoption of CSS3 has provided many possibilities for creating beautiful visual effects without using image files. Here is an example: on an early iteration of the design of the various headings on GoToQuiz, I used this sprite combined with CSS to create the rounded color bar appearance.
Simple, Light-Weight Radio Button Replacement jQuery Plugin
I was looking for a jQuery plugin to replace the standard browser radio buttons with custom images, but I had a few requirements. 1) Tabbing and focus had to function correctly, and 2) the javascript had to be light-weight. After looking into some heavy plugins and some simple ones that didn’t handle tabbing/focus properly, I decided to roll my own.
Here’s the plugin code, with comments:
(function($) { // source: http://www.gotoquiz.com/web-coding/ 'use strict'; //the plugin $.fn.niceRadio = function(options) { if (this.length !== 0) { //initialize settings & vars var settings = $.extend( { baseClass: 'nr_base', focusClass: 'nr_focus', checkedClass: 'nr_checked'}, options), $radios = this, $aset = $(); $radios.each(function(i, el) { var $radio = $(el), //create an anchor per radio that will be our radio replacement $a = $('<a href="#" id="nr_a'+ i +'" data-nr-id="'+ i +'" class="'+ settings.baseClass +' nr_for_'+ $radio.attr('name') + '" tabindex="-1"></a>'); //give each radio a custom class with a counter (nr_r#) for easy lookup $radio.addClass('nr_r'+ i) //store the counter value as data as well .data('nrId', i) //insert the anchor immediately after the radio .after($a) //hide the radio visually .css({position: 'absolute', 'margin-left': -2999}); //if radio was already checked, add the checked class to it's replacement if ($radio.is(':checked')) $a.addClass(settings.checkedClass); //collect the set of anchors as they are created $aset = $aset.add($a); }); //when an anchor is clicked... $aset.click(function(e) { e.preventDefault(); //$clicked is the anchor var $clicked = $(this), //$checked is the radio to be checked $checked = $('.nr_r'+ $clicked.data('nrId')); //check it $checked.prop('checked', true) //focus it .focus() //trigger the change event on it .change(); }); //when a radio received focus, add the focus class to its replacement $radios.focus(function() { $('#nr_a'+ $(this).data('nrId')).addClass(settings.focusClass); }) //when a radio loses focus, remove the focus class from it's replacement .blur(function() { $('#nr_a'+ $(this).data('nrId')).removeClass(settings.focusClass); }) //when a radio is changed, call the changeEvent function .change(function() { changeEvent($(this)); }); return $radios; } //called when a radio selection changes function changeEvent($radio) { //if $radio is checked... if ($radio.prop('checked')) { //remove checked class from any other anchor associated with //radios of the same name $('.nr_for_'+ $radio.attr('name')).removeClass(settings.checkedClass); //add checked class to this radio's replacement $('#nr_a'+ $radio.data('nrId')).addClass(settings.checkedClass); } else $('#nr_a'+ $radio.data('nrId')).removeClass(settings.checkedClass); } }; })(jQuery);
This requires some CSS rules, which might look like:
.nr_base { display: inline-block; height: 16px; width: 16px; background: transparent url(radio_sprite.png) no-repeat 0 0; vertical-align: middle; } .nr_checked { background-position: -16px 0; } .nr_focus { outline: 1px dotted #555; } .nr_checked.nr_focus { outline: none; }
Minified, this radio replacement plugin shrinks to under 1KB. It handles tabbing and focus properly, for both screen readers and keyboard users. It handles radio changes via clicking associated labels. And by using an anchor tag to hold the radio replacement image, it handles the odd browser that does not treat labels as clickable (a problem with some of the other radio replacement plugins I encountered).
Feel free to use this plugin as you like, and I hope it proves useful! Let me know if you find any bugs or particular browser quirks.
Really Cool Glossy Progress/Result Bars in Pure CSS
Inspired by this progress bar design, I decided to recreate the look using CSS. The end result is this:
The bars above were created purely with CSS (taking advantage of CSS3 properties such as linear-gradient and box-shadow), supported by all recent browsers versions except IE9 and below. Here is how, beginning with the HTML code.
<div id="container"> <p id="title">Here are the results:</p> <p>Option 1</p> <!-- .barOuter is the box containing the bar, .barInner is the bar itself --> <div class="barOuter"><div class="barInner" style="width: 53%;"></div><div class="barLabel">53%</div></div> <p>Option 2</p> <div class="barOuter"><div class="barInner" style="width: 24%;"></div><div class="barLabel">24%</div></div> <p>Option 3</p> <div class="barOuter"><div class="barInner" style="width: 91%;"></div><div class="barLabel">91%</div></div> </div>
Selecting Rows With Chinese Characters in MySQL
…or Russian/Cyrillic characters, or Arabic, or any characters of non-latin origin.
On this blog, I’ve let tons of spam comments pile up in the pending queue. Their number grew to the point of being completely unmanageable. Finally I decided to clean it up, and I had to find a way to bulk delete rows from the wp_comments table in the WordPress database. One thing I noticed was that hundreds, probably thousands, of spam comments contained Chinese characters. Other comments contained Russian or other non-latin characters. But how to select comments in which non-latin characters appear without selecting for specific strings (for example by using LIKE or REGEXP)?
It turns out the way to do it is like this:
SELECT * FROM wp_comments WHERE comment_content != CONVERT(comment_content USING latin1)
CONVERT … USING will convert the column value to the specified character set, replacing any characters that do not map into the set with a question mark. If there are no Chinese/Russian/etc. characters, the output of CONVERT … USING will be the same as the unconverted column value.
This was the SQL statement I used to delete the unwanted posts:
DELETE FROM wp_comments WHERE comment_content != CONVERT(comment_content USING latin1) AND comment_approved = '0'
If there is any interest, I can post some other SQL statements I used to clean up the spam.
Log SQL Statements With Parameter Values Filled In (Spring JDBC)
If you’re using parameterized queries with Spring JDBC (and why wouldn’t you?), it’s easy to log them with a simple log.debug()
statement. But your logged statements will be full of ?’s instead of the values, which makes them much less useful. Suppose you’d like to know what was substituted for those question marks. It’s not so difficult if you use AspectJ. Here is what is needed:
- aspectjrt.jar and aspectjweaver.jar from here.
- An aspect with a pointcut on the JdbcOperations interface, which
JdbcTemplate
implements. @Before
advice that intercepts the execution ofJdbcTemplate
methods and logs de-parameterized SQL statements.- Configuration of Spring applicationContext.xml to get it working.
Let’s start off with our class:
@Aspect public class SqlLogger { private static final Logger log = LoggerFactory.getLogger(SqlLogger.class);
Here, I’m using an org.slf4j.Logger
, but any logging framework will work.
Next step is to add the method that will capture the SQL and parameters as they are executed. Again, I’m using Spring JDBC, so all calls are made to an object that implements JdbcOperations
. We can set up our class to spy on all relevant calls by defining our method like so:
Make Rainbow-Colored Lists (and other HTML elements) – jQuery plugin
How to make rainbow-colored lists? I started searching for a solution and found a plugin for rainbow text, which inspired me to write my own jQuery plugin to rainbow-ify lists and other sets of HTML elements.
I’m used to working with RGB hex color codes. But there is no intuitive way to generate a spectrum of color using RGB, so I turned to HSL (hue, saturation and lightness). Most modern browsers support colors specified with hsl(hue, saturation%, lightness%)
, although IE versions prior to 9 do not. So I started out using HSL, adding IE support later. I found that keeping the saturation and lightness constant at 100% and 80% respectively while increasing the hue in steps produces a really nice rainbow effect.
Let’s back up a bit. Consider the following styled list:
I’d like to color this list to look like a rainbow. Using CSS alone is unwieldy; without knowing in advance the number of elements in the list, it’s impossible. We need to use Javascript to get this result:
Here’s how I did it–my rainbow plugin code (~1KB when minified):
jQuery Shake Plugin to, er, Make Elements Shake
I wanted to make a submit button shake back and forth when the form input is invalid, signaling the user that there was a problem. Simple enough. So I googled around for an existing solution, but the jquery plugins I found only made elements vibrate like crazy. I wanted a left-right shake.
Well it turned out to be very easy to implement myself. Minified, my shake jquery plugin is less than 0.5KB. Here’s a demo page. Here’s the code:
(function($) { 'use strict'; //the plugin call $.shake = $.fn.shake = function(options) { if (this.length !== 0) { //initialize settings & vars var settings = $.extend( { distance: 12, iterations: 3, duration: 160, easing: 'swing'}, options), $shakeThis = this; return $shakeThis.each(function(i, el) { var $el = $(el), marg = ($el.parent().width() / 2) - ($el.outerWidth() / 2); $el.css('margin-left', marg); for (var j = 0; j < settings.iterations; j++) leftRight($el, marg); $el.animate({'margin-left': marg}, settings); }); } function leftRight($el, marg) { $el.animate({'margin-left': (marg - settings.distance)}, settings) .animate({'margin-left': (marg + settings.distance)}, settings); } }; })(jQuery);