Relationship Breakdown im scared to be alone

Im 47yo male and i am scared to be alone. For years i have struggled to fit in and feel accepted. I have been on and off depression medications over the years. I have never been married and have no kids. For a number of reasons i am estranged from my immediate family and feel alone in this world and have for years. A couple of years ago i meet my current partner who is a lot younger than me. 25yr age gap and before anyone jumps to conclusions the relationship is not a financial one. Please don't judge me nobody can help who they fall in love with. Things have been amazing and we have both been very happy. Now my partner wants to experience and experiment other things life has on offer. I have never wanted him to ever miss out on anything and want him to be happy. I do not and have never told him what to do. I have agreed (even though it hurts) to a sort of open relationship even though it is against my values. I did so because i didnt want to lose him. We currently live together and now he wants to move in and share a house with some of his friends. He says he wants to still be in a relationship with me. I am scared that i wont see him or he wont make time for us. I am so scared that i will go back to the loneliness I had before I met him and i dont want to either. It might sound stupid but i dont have the confidence to meet people and make friends. I don't know what I should do. I am so tired of constantly feeling like I'm not good enough or deserve to be happy. Please help me. Any advice on what i should do would be appreciated.