The Old Testament Law set aside the Sabbath as a day of rest. The people in the Bible lands and times did not work or travel on the Sabbath. The legalistic Pharisees even set a specific distance (2,000 cubits) that a person could legally travel on the Sabbath day.
In practice, people did not travel far on the Sabbath. Jesus and his disciples stayed in the town of Bethany and traveled back and forth to Jerusalem on many Sabbath days. The east side of the Mount of Olives near Bethany was about two miles from Jerusalem. Certainly, many of the Judeans in Bethany would travel to the Temple in Jerusalem on the Sabbath.
The expression "a Sabbath day's journey" simply meant a short distance, such as one would normally travel on the Sabbath.