There are a few factors at play here. Is your speedo correct, some of our work cars are out by up to 6kmh from new against a gps. Why would you not pull over to the left and let him pass as he was obviously traveling faster than you when you saw him coming up behind you? I have been driving MC's for the last 30 years and I still find it most frustrating just getting up to speed and have someone in front doing a couple of k's less. There is a little thing called common courtesy that is not so common these days, if I look in the mirror and see a couple of cars close behind me I will pull over to the left when safe and let them through it is no big deal and keeps everyone happy, some people think because they pay rego they own the road.
Not trying to be offensive here just an observation from a few years on the road