Can't Build on Half of My Territory

[Vert 21]

I found a nice little spot to call home and I put my flag down. I then try to add a workbench to start building and it won't let me place it down. First it said I was not allowed to build here on this server. Later it give no message but still won't let me place the bench. I then notice that it let's me place it on the other half of my territory. The only thing near my base is a boat trader that is about 350 meters from my flag.
First, I am the server admin. I have set the distance to build territories from traders to 100 meters (temporarily to see if it had to do with my issue). There are no spawn zones and no cement mixers nearby. There is a small town about 500 meters away (edge of town, center is further away). I can build on the side facing towards the town but not the boat trader which is in the opposite direction. For the life of me I can't find any other setting to control build distance from traders, spawns, towns, etc.
The only other thing that comes to mind is the build distance to other territories, which is still set to 325 by default.
Could it be treating the boat trader as a territory? This might explain the build limitation of 350 meters I guess, but that doesn't make a whole lot of sense to me.
Any suggestions would be very much appreciated.

Thanks.

~Vert