It looks to me like you have it laid out really well around the knot hole. The grain around it should be good enough to trust.
Like JW said it definitely has loads of challenges, but that knot hole looks like one that’s workable.
Id clean up any loose undependable wood and leave the hole, but I kinda like holes.
Bjrogg