LÖVE

Trying to work out a way to detect when the mouse is being shaken. i.e. moved back and forth rapidly in a relatively small space. I thought I had figured it out but not quite.

I was using all three mouse callbacks. Pressed, Moved and Released to store the information. And was basing my detection on how far the mouse has moved compared to how much total distance it has traveled and while it worked, it also sends false positives if the shake timer reaches the end when you happen to be moving the mouse in the area of its initial press.

I'm probably missing a pretty simple solution involving maths or something.

This is how I imagine it working:

The 2 circles represent boundaries of mouse movement. Obviously these could be experimented with and resized.
The red one represents an error boundary- this way, small, unintentional movements won't trigger anything.
Once the mouse is moved outside of the red circle, add a count. Redo this process on each update. Once the count reaches a certain number (say, 3) then a shake could be detected. The blue circle could be a max distance.

I have no idea if this will work, but that's how I would try it.

I wrote a small program, that should do the trick.

The idea behind it: If you are in an small enough area, then you are really shaking and not just moving the mouse around and getting back to the starting point. In my case I just made an rectangular approximation, but it does the job.

I'd personally base shake detection on 3 parameters;
1. the speed of the mouse,
2. the direction of the movement, (coupled with the above would make a velocity vector)
3. time restrictions. (too fast jitters and too slow movement wouldn't count as shaking)
(4. and another that basically says at least how many movements constitute a shake)

In mousemove, compute dx,dy, from those a magnitude and angle, and save those with a timestamp into a table, acting as a buffer with some length (#4).
Check between all values in the table, whether the timestamp difference is between your bounds for detection (#3), whether the mouse moved enough (#1), and whether the difference between angles is big enough, let's say >90° and <270°(#2).
If all of these three criteria are met for all value pairs, you did a shake.

Of course, tweaking it is the hard part, to suit your needs