OW,
Many skimmers come with a 'knockout' plug so that the hole can be used as an overflow drain. In theory, the hole is at the right spot to keep the water level where is should be. Depending on how things are built, that may, or may not, be the case..
I use that hole as a fill line for my pool and have found that I like to fill the pool an inch or so above the hole. Since it is a fill line, I don't have to worry about it draining my pool, but I plug it anyway, just to keep the crud out, when not in use.
Only you can tell if the hole is keeping your pool at the right level.. The water level should be about the middle of the tile line. I like to start off about an inch above that, so that I have longer between fill-ups.
Some pool builders are pretty lazy and just run a pipe off the hole and into the ground.. Sigh..
Thanks,
Jim R.