Fillers should not affect the cure time at all as Matt said.
The only time they might is if there is some contaminant in them but if you have collected you own carefully this should not be an issue.
A word on epoxy ratios - and I know someone will come up with an exception to this but I say it is a hard and fast rule - Always use the ratio given by the supplier for your brand of epoxy. This is not like polyester where cure time is adjusted at will by mix ratios. Most brands of epoxy are fairly tolerant of variation but tolerance is all it is - the best strength will be obtained at the recommended ratio and if the ratio is far enough off it will not cure. (This is how most of us know when we haven't mixed it right

)
The 2 main things that go wrong in the process are
- not adding the part 2 epoxy
and
- forgetting to mix after adding part 2 (I've done that once - made a hell of a mess

)
These don't happen often and you have had more than 1 crook batch so it's unlikely to be one of them but keep them in mind.
The main thing that I've found affects cure time is temperature and since we are at that time of year when seasons are changing I wonder if this is part of your problem. You don't mention where you are from but if you are in the southern hemisphere then the weather is getting cooler on average - although wherever you are there are day to day variations. Was the first troublesome mix done on a day that was cooler than the previous ones? If so this would have slowed things down and the next attempt could have been affected by varying the mix ratio.
As for filler ratios, I always mix the epoxy first then add sawdust or whatever a bit at a time until I get the right consistency for what I want to do eg filleting, gluing, end pour etc. Compared to the careful way I measure the epoxy this is really sloppy practice - but it always works.
Good luck with the build and I'm sorry if I've just told you a heap of stuff you already know
