My view is that the non-Jewish neighbors were within walking distance but did not share wells, fields, gardens, pastures, etc. In addition to the gathering of metals and jewels, they had to be reasonably close with their neighbors because:
- There was intermarriage.
- The Erev rav joined them in the hours after the 10th plague and before they left the next morning. Even if these were only family members of those who intermarried, they had to have enough time to figure out what was happening and prepare to leave (which can be quick but requires travel both ways).
- Moses and Aaron travelled multiple times to the palace to meet with Pharaoh. They probably traveled by boat (which was common in that era) which is quicker that walking or on a pack animal (or even a horse, if they were available to them, which is unlikely). We don't know how long this journey took but the text indicates new locations "the following day." If the palace* was less than a day's journey, other slave villages, regular workers, and others wouldn't be too far. They also talk about going out of the city, which implies the palace is inside the city, and not next to the edge.
- There is talk of trade. Purchasing the lamb for Passover, for example. This could be a purchase from other Jews, but it implies there is trade among neighbors.
- Their main work is to make bricks and to tend Pharaoh's fields. This work is commute distance from their villages and the output of that work would need to be easily transportable to the city and elsewhere (both are done by the river).
I imagine the edge of the Hebrew villages to be not far from the part of the river with the brickyard, as well as various fields. Down river (north/northeast) would be the city and palace. Back behind, around, and in-between the villages would be the undamaged gardens belonging to the Hebrews, as well as their pastures and livestock.
There would also be other villages for slaves and/or workers and the Hebrews would have interacted to some degree with them. It is also possible that the brickmaking and fieldwork was shared. The Hebrews had their own quotas but that doesn't mean the entire brickyard was only them. I don't know one way or another, but it's a possible way for people to meet.
The Nile river branches in the Delta region, which is pretty much everything north of modern day Cairo. So the river is not north-south in Goshen. If the Hebrews are, for example, southeast of the river, they would not extend alongside it. Next to the river would be fields all the way up to Nubia and beyond ("up" in Egypt means against the flow of the river, which flows from south to north). The Hebrews would be in one of the most eastern branches, to the east of that branch (so they do not have to cross it to leave towards Sinai).

- = There were several palaces and the court moved around.