Agentgogo CRM does need to fully connect to your gmail or gsuite account (if that's the type of email address you're using) and the connection process and access levels requests it needs are standard across all applications that connect to gmail. In other words, this is normal and safe.
It does read your emails in a sense, but all it is doing is looking for replies from contacts in your Agentgogo CRM. It's not scanning each email or reading the contents of them.
The connection is also what makes it possible to send emails directly from your gmail account.
Please also note that during the connection process when you initially connect you may see the warning that the connecting app (called Nylas) is unsafe. This is not true, but we chose not to pay the $70k fee to have google remove that warning so we could keep the costs lower for you.