To delete a connection, first select it and then click on the trash can icon. You can also drag and drop the connection you want to delete on the trash can icon.
Connections deletion is not undoable, which means that once you delete a particular connection, it will be permanently lost.