I suppose it is less of a concern while in the water but I would put them back in just to keep things out of them and clogging them like bees, mice, sea weed, etc...
I put a large first aid kit up there. but it wouldn't be too hard, use starboard vertically to make your divider and then if space allows thin strips to separate your tackle boxes so you don't have to take them all out to get the bottom one.