Ensure you are using a browser which supports folders. Navigate to the desired library

Click Upload

Select Folders from the dropdown menu. You will not get this option in other browsers (e.g. IE11) 

Select the folder if available) you wish to upload

At the bottom of Windows Explorer click OK.

To overwrite existing files with the same name click on the message in the top right of the task bar

Click Replace in the Progress Pane