But apparently there there is a design flaw in MySQL 5.0 and before with respect to how it handles indexes that caused dropping indexes on tables very slow, and also adding a new index on the table slower than it needed to be. Gory details here: http://bugs.mysql.com/bug.php?id=2364

But in 5.1, at least dropping table ought to be near instantaneous. Adding an index? Umm… it seemed the old behaviour was:

Given the same MyISAM table T having four indexes (ndx1,ndx2,ndx3,ndx4)
and you want to ‘alter table T add index ndx5 (…);’ here is exactly
what happens (from: http://lists.mysql.com/mysql/202489)

1) MySQL copies T.MYD to a temp table, i.e., S.MYD and a zero byte S.MYI.
2) MySQL does ‘alter table S add index ndx1 (…);
3) MySQL does ‘alter table S add index ndx2 (…);
4) MySQL does ‘alter table S add index ndx3 (…);
5) MySQL does ‘alter table S add index ndx4 (…);
6) MySQL does ‘alter table S add index ndx5 (…);
7) MySQL deletes T.MYD and deletes T.MYI
8) MySQL renames S.MYD to T.MYD, and renames S.MYI to T.MYI

Note that each of the “alter table” step involves copying the table to a temp table with no indexes, applying all existing indexes one by one, and then applying the new index…. so in the above case:

In fact, for table T with no indexes and you want to add N indexes
MySQL will copy the MYD N times
MySQL will copy the MYI N times
MySQL will run ‘alter table add index’ N(N+1)/2 times if adding an index
MySQL will run ‘alter table drop index’ N(N-1)/2 times if dropping an index

So here is a better way to add indexes to a table:

1) create table T1 like T;
This creates an empty table T1 with indexes ndx1,ndx2,ndx3 and ndx4.
2) alter table T1 drop index ndx3;
This drops index ndx3 on the empty T1, which should be instantaneous.
3) insert into T1 select * from T;
This will populate table T and load all three(3) indexes for T1 in one pass.
4) drop table table T;
5) alter table T1 rename to T;

Last night, when I needed to quickly download something, though, it felt very slow. Was it the site I was downloading from? Or was it starhub? I connected to speedtest.net and foudn that my download speed was below 2Mbps! I did the test after bypassing my home router, of course. The download site could be slow. Speedtest.net might be slow. Or starhub could be slow (just for a little while)

I did the test again morning and here is what I got:

Hmm.. not much improvement. Where’s my 30Mbps, pal?

I am thinking of doing the test several times over the course of the week, and see whether I am really getting my money’s worth. I will downgrade to a slower link and see what that gives me, as well, if the bandwidth does not improve soon. By the way, anyone know someone at Starhub who can give a firm answer about what “upto 30Mbps
download speed” really means in practice?

The FCC in the US does not think it means anything:
Recently, there was a story about American ISPs exaggerating their connection speeds. The FCC plans to do it own verification of these claims, and introduce a new labeling format for connection speeds that might be closer to reality and also tell consumers more accurately what kind of speed they will see in practice. Perhaps Singapore’s IDA could some the same?

2010-09-10 16:30 SGT

2.96 Mbps:

2010-09-11 00:25 SGT
2.16Mbps:

Much higher download speed of 10Mbps from server in Cyberjaya, Malaysia:

2010-09-17 01:40am

Repeating the same test from my desktop, connected via 1Gbps ethernet to the router:

Singapore: 16Mbps

CyberJaya: 15Mbps

2010-09-17 23:16

Singapore: 1.84Mbps

CyberJaya: 5.8Mbps

I have a value-object:

public class Datum
        implements Serializable
    {
    private int id;
    private String name;

    public Datum(int id, String name)
        {
        this.id = id;
        this.name = name;
        }

    public int getId()
        {
        return id;
        }

    public String getName()
        {
        return name;
        }

    @Override
    public String toString()
        {
        return "Datum{" + "id=" + id + "name=" + name + '}';
        }

    }

I have a service that produces Datum objects. The service interface is:

public interface IDatumProducer
    {
    public Datum produce();
    }

and a sample implementation might be:

public class RandomDatumProducerImpl
        implements IDatumProducer
    {
    private static final Logger log =
            LoggerFactory.getLogger(RandomDatumProducerImpl.class);

    private Random rnd = new Random(System.currentTimeMillis());

    @Override
    public Datum produce()
        {
        int i = rnd.nextInt();
        Datum datum = new Datum(i, String.valueOf(i));
        log.info("Produced {}", datum);
        return datum;
        }

    }

Here is a client for the service:

public class DatumProducerClient
    {
    private static final Logger log =
            LoggerFactory.getLogger(DatumProducerClient.class);

    private final IDatumProducer producer;

    public DatumProducerClient(IDatumProducer producer)
        {
        this.producer = producer;
        }

    @PostConstruct
    public void init()
        {
        while (true)
            {
            try
                {
                Datum datum = producer.produce();
                log.info("Got {}", datum);
                Thread.sleep(1000L);
                }
            catch (InterruptedException ex)
                {
                log.error("Interrupted", ex);
                }
            }
        }
    }

Now I want to run the DatumProducerClient and the IDatumProducer on different machines. The client should connect remotely to the producer, invoke the produce method, and do its stuff.

We can expose the service easily using Spring. The spring remoting (JMS) manual section gives the spring configuration XML to do this, but no example for doing this in Java Config. Turns out there is one small detail to take care of when using Java config.

Producer (Server)

Here is the application config for the producer side:

@Configuration
public class ProducerConfig
    {

    @Bean
    public AbstractMessageListenerContainer messageListenerContainer()
        {
        SimpleMessageListenerContainer container =
                new SimpleMessageListenerContainer();
        container.setConnectionFactory(jmsConnectionFactory());
        container.setDestination(jmsQueue());
        container.setMessageListener(jmsInvokerServiceExporter());
        container.setConcurrentConsumers(3);
        return container;
        }

    @Bean
    public JmsInvokerServiceExporter jmsInvokerServiceExporter()
        {
        JmsInvokerServiceExporter serviceExporter =
                new JmsInvokerServiceExporter();
        serviceExporter.setService(datumProducer());
        serviceExporter.setServiceInterface(IDatumProducer.class);
        return serviceExporter;
        }

    @Bean
    public ConnectionFactory jmsConnectionFactory()
        {
        ActiveMQConnectionFactory baseConnectionFactory =
                new ActiveMQConnectionFactory();
        baseConnectionFactory.setBrokerURL("tcp://localhost:61616");
        return baseConnectionFactory;
        }

    @Bean
    public Queue jmsQueue()
        {
        ActiveMQQueue queue = new ActiveMQQueue("dataChannel");
        return queue;
        }

    @Bean
    public IDatumProducer datumProducer()
        {
        return new RandomDatumProducerImpl();
        }

    }

And here is a main method that kicks off our application.

public class PublisherMain
    {
    public static void main(String[] args)
        {
        AnnotationConfigApplicationContext ctx =
                new AnnotationConfigApplicationContext(ProducerConfig.class);
        }
    }

Consumer (Client)

Here is how the consumer is wired up.

@Configuration
public class ConsumerConfig
    {

    @Bean
    public DatumProducerClient datumProducerClient()
        {
        return new DatumProducerClient(datumProducer());
        }

    @Bean
    public IDatumProducer datumProducer()
        {
        // producerFactory.getObject() will return null if we don't define
        // it as a separate bean; if we copy over the code from
        // jmsInvokerProxyFactoryBean here, it won't work: the proxy won't
        // get "enhanced" by spring, and its getObject() method will always
        // return null.
        JmsInvokerProxyFactoryBean producerProxy = jmsInvokerProxyFactoryBean();
        return (IDatumProducer) producerProxy.getObject();
        }

    // We can't fold this code into datumProducer above
    // The proxy bean must be defined separately, otherwise spring won't get
    // a chance to enhance it.
    @Bean
    public JmsInvokerProxyFactoryBean jmsInvokerProxyFactoryBean()
        {
        JmsInvokerProxyFactoryBean producerProxy = new JmsInvokerProxyFactoryBean();
        producerProxy.setServiceInterface(IDatumProducer.class);
        producerProxy.setConnectionFactory(jmsConnectionFactory());
        producerProxy.setQueue(jmsQueue());
        return producerProxy;
        }

    @Bean
    public ConnectionFactory jmsConnectionFactory()
        {
        ActiveMQConnectionFactory baseConnectionFactory =
            new ActiveMQConnectionFactory();
        baseConnectionFactory.setBrokerURL("tcp://localhost:61616");
        return baseConnectionFactory;
        }

    @Bean
    public Queue jmsQueue()
        {
        ActiveMQQueue queue = new ActiveMQQueue("dataChannel");
        return queue;
        }
    }

and here is how the main method on the client side looks like:

public class ConsumerMain
    {
    public static void main(String[] args)
        {
        AnnotationConfigApplicationContext ctx =
                new AnnotationConfigApplicationContext(ConsumerConfig.class);
        }
    }

The important bit to note here is that the JmsInvokerProxyFactoryBean is configured as a bean in its own right. This allows spring to “enhance” it at runtime to do all the proxying for us. If this were not a separate bean, things would not work. For e.g., the following code won’t work:

// Wrong way to use a ProxyFactoryBean
    @Bean
    public IDatumProducer datumProducer()
        {
        JmsInvokerProxyFactoryBean producerProxy = new JmsInvokerProxyFactoryBean();
        producerProxy.setServiceInterface(IDatumProducer.class);
        producerProxy.setConnectionFactory(jmsConnectionFactory());
        producerProxy.setQueue(jmsQueue());
        return (IDatumProducer) producerProxy.getObject(); // Wrong: getObject always returns null
        }

That’s all!

I suppose I was mentally prepared to fight the software. If it actually
was performing such a nice and useful function, it ought to be
complicated and self-important. “Read my README”, I imagined it
saying. “Then read the INSTALL.txt, and Getting Started, and then
Adminstrator’s Guide…”. I was hoping to wing it. I’d try to start
it, see what broke, try to configure that, see what else broke, try to
fix that, and so on until I won and the software lost the game of
Yet-Another-Installation-Nightmare.

My fears turned out to be grounless.
Installing it was a breeze. One just unpacked the
software and ran a start script. All dependencies were already included
so I did not have to hunt down any. It just started up, presenting a
web-based management UI. It was already configured with sensible
defaults to get a new user up to speed quickly. Wow!

My next step was to create a user account. I brought up the web
management console in firefox. I clicked on the “Users” link, and was
presented with a list of existing user accounts. I could not see how
to create a new user. I could see how to attach users to roles, I
could see how to search for users using their name, role, email etc.
But for the life of me I could not figure out how to create a new
user account.

When I was just about to admit defeat and turn to the “User Manual”, the
answer jumped out of the page and smacked me in the forehead. There it
was, clear as day, a button labeled “Create New User”. I had been
staring at various links and trying everything on that page, expecting
that this step would be as easy as the install part was, and getting
slightly annoyed that it wasn’t. I had been clicking at everything but
this button. In fact, I had given this button a good long stare and
looked around it to find a link that I could click.

Had my brain suddenly turned into putty? What is so hard about clicking
a dar***d button?

There is nothing hard about clicking a button. Except that the button
in question did not look like a button. The button looked like a
flat, light gray box. When I hovered the mouse over the box, the cursor
did not change into something that indicated “click me”. Basically,
someone had gone out of their way to create a CSS style that made
the button obscure.

Why?

Because it looked “better”? Because it was there to be styled? Just
because? Beats me. The point is that this beautifully useful and easy
to setup software had a GUI blooper right there in the management
console on one of the most commonly used functions. In other words,
this very useful software, which was insanely great in so many other
ways, had a usability issue.

Same day. Another web page. This was a commercial web site that served
up search results arranged alphabetically. There was a nice, helpful
row of navigation links at the top, labeled “A”, “B”, and so one all the
way to “Z”, to allow the user to see results starting with that lette.
The letters were surrounded in a nice, colored box that became highlighted
when your mouse hovered over the box.

So I moused over to the box containing “P” and when it was highlighted,
I clicked.

Nothing happened.

So I double-checked and found that while the box gets highlighted when
your mouse hovered over it, only the letter itself was clickable, not
the box. The “target area” presented by the letter is significantly
smalled than the box, forcing you to be precise about where you click.
There might as well not have been any box. In fact, the box was
downright misleading. It looked good. But it lulled my busy brain into
trying to click something that was not clickable. Another usability
issue.

Usability has been on my mind recently. (Gee, can you tell?). It is
amazing that there is so much software and so many websites out there
with small, niggly usability issues that are sucking away seconds,
minutes or hours of their users life as they grumpily are trying to get
their jobs done. It is sad, really. The code is there. It works. But
the path to executing that code is strewn with brambles and thickets of
usability issues.

Programmers need to pay attention to the usability of their UIs. Most
books, tutorials and manuals focus on making the UI look good, but few
point out how to make the UI more usable. Most people don’t read the UI
design guidelines of the platform they are working on. And there are
probably no standard guidelines for designing for the web. Web
technologies are changing too fast anyway for any technology focused
guidelines to remain relevant for long. The core principles are taught
in the discipline of Interaction Design. But I don’t think many people
know what Interaction Designers do. I myself didn’t until very
recently.

There really ought to be more interaction designers out there and they
ought to have more work. Programmers and managers should be paying
attention to the usability of their software and its interface.

But for now, I am too busy slapping together functionality to pay
due attention to how it will end up getting used. Oh, I think about
this issue and try to do something about it, for sure. I try to design
the software, imagining the end user and his/her goals in mind.

But that is not enough. User interviews, mock-ups, usability testing
are some of the tools that are involved in producing usable software. I
did not budget for it; I did not set up any time for it. And I wonder
if anyone would understand me if I told them I wanted to take a flight
to meet our users (internal business folks) and do a usability testing.

Or maybe they would. One of these days, I am going to try.

]]>