I took care of my mom for a couple years while also working full time. I live 2-1/2 hours away and would come home weekends. I left in late 2020 to work from home (during covid). My brother and SIL (who live near mom) took over mom's care. I've not seen her much given hers and my covid concerns. Things are easing up; she came here for Christmas but I've not seen her since. I'm still working from home. Her birthday is 4/28. I asked SIL if there were birthday plans and she said yes, birthday dinner at 4 pm on Saturday 4/30. I thought I would drive down and bring my daughter and granddaughter and we all could have birthday dinner together. My daughter can't leave work till 2; that puts us at the restaurant at 4:30 or 5. I asked if this was okay. Response: "So why don't you just come when you can without any rush. Just arrange with your mom the timing, pick her up, and take her where it is good with you. We all can't make this dinner and we'll do something else on our own." I did not intend to impede their plans, or demand an unreasonable change (other than get there at 4:30 or 5), but I did not see why going a bit later at 4:30 or 5 would be a problem. I decided that we drive over the next day (Sunday). There's more time (for my daughter) but more difficult on me since we will have to drive over and return the same day. I sent my SIL a note and apologized for inferring that I wanted them to move the time a bit to accommodate us, and explained we are coming Sunday instead.
Try to give them the benefit of the doubt. It could be that the uncertainty of what time to leave would start things off on the wrong foot. Maybe they were concerned that the doubling of the size of the group would be too much. Perhaps she has occasional issues with sun-downing and 4:00 is a lot different than 5:00ish. Let the note suffice and move on.
If you prefer driving the day before, go ahead and do so. You can make it a Sunday brunch and get back home before dark.