Prior to this commit, most flows that ended in a long HTTP request to an LLM provider would be routed through the main.py controller except for the `get_direct_response` which was called in an orm call to the ai_agent model. This is problematic because if we want in the future to re-route these calls to the GEVENT server, we couldn't just use the /ai/* path.

In this commit we create a new controller endpoint for the `get_direct_response` method and replace the ORM call instances with RPC calls to that endpoint. We also rename the two controller files from `agent.py` and `main.py` to `ai.py` and `thread.py` respectively, to better adhere to the new naming standards.

Removing the `get_direct_response` method from the `ai.agent` model meant that it couldn't be used directly from the back-end.

Current solution makes the `get_direct_response` only a controller accessible method and turns the `_generate_response` backend method as the method to call directly from the backend.

Also the markdown to HTML conversion and the HTML sanitization are now done directly in the end of the `_generate_response` method instead of at the `get_direct_response` and the `_post_ai_response` methods

Also, changes to the test files were performed to reflect changes in the business code.

Because formatting is now done in the `_generate_response` function, we change the patched methods. Instead of patching generate_response, we now patch only the actual `_request` method.

For that, we made some modifications to `test_llm_tool_calling.py` so the mock_request method is now public and we can use it in other files.

We also add a sudo when fetching tools inside `_generate_response` since manual template rendering is not done with sudo, and if done by a non admin user, it will crash. (Non-admin users don't have access to tools).

Task-5130087