Are we tiny?#
gptme is intended to be small and simple, and focus on doing the right thing in the right way, rather than all the things in all the ways.
The benefits of this approach are many:
It is easier to understand and maintain.
It is easier to contribute to.
It is easier to learn.
It is easier to extend.
It is more fun to work on.
Being aggressive about keeping things small and simple is a way to keep the project maintainable and fun to work on. The fastest way to kill a project is to make it too big and complex, and suffer burnout as a result.
Another major benefit of keeping things small and simple is that it makes it easier for AI to understand and work with the codebase. This is a major goal of the project, and it is important to keep in mind that the simpler the codebase is, the easier it will be for AI to work with it:
“The simpler your API is, the more effectively the AI can harness it when generating code.”
—Kenneth Reitz (and many others)
To that end, in this document we will present some statistics about the current state of the project, trying to be mindful to keep an eye on this page and make sure we are not growing too much.
Startup time#
$ make bench-importtime
...
207520 | openai.types
254743 | gptme.chat
372040 | openai
407984 | gptme.cli
461461 | gptme
real 0m1.375s
user 0m1.207s
sys 0m0.182s
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
Lines of code#
LoC Core#
$ make cloc-core
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc gptme/*.py gptme/llm/*.py gptme/util/*.py gptme/tools/__init__.py gptme/tools/base.py --by-file
github.com/AlDanial/cloc v 1.98 T=0.06 s (643.5 files/s, 142976.9 lines/s)
---------------------------------------------------------------------------------
File blank comment code
---------------------------------------------------------------------------------
gptme/llm/llm_openai.py 97 52 387
gptme/chat.py 80 112 383
gptme/llm/llm_anthropic.py 86 67 362
gptme/tools/base.py 88 77 352
gptme/logmanager.py 74 66 326
gptme/cli.py 55 50 296
gptme/util/prompt.py 72 96 281
gptme/message.py 58 65 239
gptme/ncurses.py 39 5 238
gptme/commands.py 18 19 232
gptme/llm/models.py 29 15 232
gptme/util/context.py 63 77 232
gptme/llm/__init__.py 52 44 218
gptme/util/ask_execute.py 48 44 191
gptme/util/cli.py 51 31 174
gptme/util/__init__.py 37 22 158
gptme/prompts.py 58 83 157
gptme/llm/llm_openai_models.py 2 10 138
gptme/tools/__init__.py 41 17 138
gptme/config.py 36 14 123
gptme/util/export.py 17 25 86
gptme/init.py 25 19 76
gptme/util/reduce.py 24 32 74
gptme/util/generate_name.py 5 6 69
gptme/codeblock.py 13 10 64
gptme/dirs.py 27 14 55
gptme/__version__.py 9 7 45
gptme/util/cost.py 12 20 42
gptme/util/clipboard.py 8 2 35
gptme/util/terminal.py 23 21 34
gptme/wut.py 13 12 31
gptme/util/useredit.py 12 13 21
gptme/util/interrupt.py 13 14 20
gptme/constants.py 7 11 13
gptme/__init__.py 1 0 8
gptme/__main__.py 1 0 3
---------------------------------------------------------------------------------
SUM: 1294 1172 5533
---------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
LoC LLM#
$ make cloc-llm
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc gptme/llm/*.py --by-file
github.com/AlDanial/cloc v 1.98 T=0.02 s (306.2 files/s, 109688.2 lines/s)
---------------------------------------------------------------------------------
File blank comment code
---------------------------------------------------------------------------------
gptme/llm/llm_openai.py 97 52 387
gptme/llm/llm_anthropic.py 86 67 362
gptme/llm/models.py 29 15 232
gptme/llm/__init__.py 52 44 218
gptme/llm/llm_openai_models.py 2 10 138
---------------------------------------------------------------------------------
SUM: 266 188 1337
---------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
LoC Tools#
$ make cloc-tools
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc gptme/tools/*.py --by-file
github.com/AlDanial/cloc v 1.98 T=0.04 s (581.0 files/s, 138371.4 lines/s)
-------------------------------------------------------------------------------------
File blank comment code
-------------------------------------------------------------------------------------
gptme/tools/base.py 88 77 352
gptme/tools/computer.py 101 153 346
gptme/tools/shell.py 74 68 317
gptme/tools/tts.py 114 124 316
gptme/tools/tmux.py 52 72 211
gptme/tools/patch.py 44 60 195
gptme/tools/_browser_playwright.py 60 20 185
gptme/tools/python.py 56 49 172
gptme/tools/save.py 32 38 167
gptme/tools/chats.py 43 49 142
gptme/tools/rag.py 61 75 141
gptme/tools/__init__.py 41 17 138
gptme/tools/subagent.py 37 29 102
gptme/tools/_browser_thread.py 20 3 97
gptme/tools/browser.py 31 63 64
gptme/tools/vision.py 19 12 53
gptme/tools/screenshot.py 11 9 34
gptme/tools/_browser_lynx.py 9 13 30
gptme/tools/youtube.py 10 1 26
gptme/tools/gh.py 14 21 16
gptme/tools/read.py 6 8 13
-------------------------------------------------------------------------------------
SUM: 923 961 3117
-------------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
LoC Server#
$ make cloc-server
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc gptme/server --by-file
github.com/AlDanial/cloc v 1.98 T=0.01 s (711.4 files/s, 112751.5 lines/s)
------------------------------------------------------------------------------------
File blank comment code
------------------------------------------------------------------------------------
gptme/server/static/main.js 35 45 316
gptme/server/api.py 61 50 241
gptme/server/static/index.html 11 15 148
gptme/server/static/style.css 27 4 143
gptme/server/static/computer.html 1 1 90
gptme/server/cli.py 8 5 55
gptme/server/__init__.py 2 3 3
gptme/server/__main__.py 1 0 3
------------------------------------------------------------------------------------
SUM: 146 123 999
------------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
LoC Tests#
$ make cloc-tests
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc tests --by-file
github.com/AlDanial/cloc v 1.98 T=0.03 s (1089.8 files/s, 118861.6 lines/s)
------------------------------------------------------------------------------------------
File blank comment code
------------------------------------------------------------------------------------------
tests/test_cli.py 79 41 298
tests/test_llm_openai.py 28 0 244
tests/test_llm_anthropic.py 26 0 221
tests/test_tool_use.py 8 18 127
tests/test_prompt.py 35 36 120
tests/test_util_cli.py 32 23 94
tests/test_server.py 28 10 84
tests/test_shell.py 27 47 82
tests/test_tools.py 48 6 82
tests/test_prompt_tools.py 5 4 78
tests/test_tools_tts.py 29 37 60
tests/conftest.py 20 12 58
tests/test_context.py 18 13 58
tests/test_tools_patch.py 36 91 54
tests/test_eval.py 8 9 49
tests/test-integration.sh 22 36 43
tests/test_message.py 7 17 39
tests/test_reduce.py 10 2 39
tests/test_codeblock.py 14 40 37
tests/test_util.py 28 51 37
tests/test_tools_rag.py 14 7 35
tests/test_chat.py 17 28 34
tests/test_tools_python.py 19 1 26
tests/test_browser.py 14 14 18
tests/test_logmanager.py 8 5 18
tests/test_prompts.py 8 2 16
tests/test_tools_chats.py 2 0 13
tests/test_tools_shell.py 9 11 13
tests/test_tools_subagent.py 7 10 11
tests/test_config.py 2 0 5
------------------------------------------------------------------------------------------
SUM: 608 571 2093
------------------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
LoC Eval#
$ make cloc-eval
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc gptme/eval/**.py --by-file
github.com/AlDanial/cloc v 1.98 T=0.01 s (615.5 files/s, 80940.8 lines/s)
-------------------------------------------------------------------------------------
File blank comment code
-------------------------------------------------------------------------------------
gptme/eval/main.py 55 22 359
gptme/eval/run.py 48 32 279
gptme/eval/agents.py 12 5 56
gptme/eval/execenv.py 10 14 55
gptme/eval/filestore.py 5 2 32
gptme/eval/types.py 13 12 32
gptme/eval/__init__.py 1 0 4
gptme/eval/__main__.py 1 0 3
-------------------------------------------------------------------------------------
SUM: 145 87 820
-------------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
LoC Total#
$ make cloc-total
make[2]: Entering directory '/home/runner/work/gptme/gptme'
cloc gptme/__version__.py gptme/server/__init__.py gptme/server/cli.py gptme/server/api.py gptme/server/__main__.py gptme/chat.py gptme/wut.py gptme/__init__.py gptme/eval/__init__.py gptme/eval/types.py gptme/eval/run.py gptme/eval/suites/browser.py gptme/eval/suites/__init__.py gptme/eval/suites/init_projects.py gptme/eval/suites/basic.py gptme/eval/main.py gptme/eval/agents.py gptme/eval/__main__.py gptme/eval/filestore.py gptme/eval/execenv.py gptme/ncurses.py gptme/llm/llm_openai.py gptme/llm/__init__.py gptme/llm/models.py gptme/llm/llm_anthropic.py gptme/llm/llm_openai_models.py gptme/constants.py gptme/config.py gptme/cli.py gptme/commands.py gptme/codeblock.py gptme/util/export.py gptme/util/__init__.py gptme/util/useredit.py gptme/util/prompt.py gptme/util/cli.py gptme/util/terminal.py gptme/util/context.py gptme/util/reduce.py gptme/util/clipboard.py gptme/util/cost.py gptme/util/generate_name.py gptme/util/interrupt.py gptme/util/ask_execute.py gptme/__main__.py gptme/prompts.py gptme/dirs.py gptme/logmanager.py gptme/init.py gptme/message.py gptme/tools/tmux.py gptme/tools/browser.py gptme/tools/python.py gptme/tools/gh.py gptme/tools/__init__.py gptme/tools/youtube.py gptme/tools/_browser_lynx.py gptme/tools/vision.py gptme/tools/base.py gptme/tools/screenshot.py gptme/tools/computer.py gptme/tools/save.py gptme/tools/patch.py gptme/tools/tts.py gptme/tools/read.py gptme/tools/_browser_playwright.py gptme/tools/chats.py gptme/tools/shell.py gptme/tools/_browser_thread.py gptme/tools/rag.py gptme/tools/subagent.py tests/test_prompts.py tests/test_llm_openai.py tests/test_shell.py tests/test_llm_anthropic.py tests/test_config.py tests/test_server.py tests/test_context.py tests/test_tools_python.py tests/test_tools_rag.py tests/test_tool_use.py tests/test_browser.py tests/test_eval.py tests/test_tools_patch.py tests/test_reduce.py tests/conftest.py tests/test_logmanager.py tests/test_codeblock.py tests/test_util.py tests/test_tools_subagent.py tests/test_tools_chats.py tests/test_prompt.py tests/test_prompt_tools.py tests/test_tools.py tests/test_tools_tts.py tests/test_tools_shell.py tests/test_chat.py tests/test_util_cli.py tests/test_message.py tests/test_cli.py scripts/auto_rename_logs.py scripts/describe_api.py scripts/summarize_project.py scripts/gpt_todoer.py scripts/list_user_messages.py scripts/shorten_details.py scripts/train/collect.py scripts/treeofthoughts.py --by-file
github.com/AlDanial/cloc v 1.98 T=0.12 s (878.5 files/s, 147040.6 lines/s)
-------------------------------------------------------------------------------------
File blank comment code
-------------------------------------------------------------------------------------
gptme/llm/llm_openai.py 97 52 387
gptme/chat.py 80 112 383
gptme/llm/llm_anthropic.py 86 67 362
gptme/eval/main.py 55 22 359
gptme/tools/base.py 88 77 352
gptme/tools/computer.py 101 153 346
gptme/logmanager.py 74 66 326
gptme/tools/shell.py 74 68 317
gptme/tools/tts.py 114 124 316
tests/test_cli.py 79 41 298
gptme/cli.py 55 50 296
gptme/util/prompt.py 72 96 281
gptme/eval/run.py 48 32 279
tests/test_llm_openai.py 28 0 244
gptme/server/api.py 61 50 241
gptme/message.py 58 65 239
gptme/ncurses.py 39 5 238
gptme/commands.py 18 19 232
gptme/llm/models.py 29 15 232
gptme/util/context.py 63 77 232
tests/test_llm_anthropic.py 26 0 221
gptme/llm/__init__.py 52 44 218
gptme/tools/tmux.py 52 72 211
gptme/tools/patch.py 44 60 195
gptme/util/ask_execute.py 48 44 191
gptme/tools/_browser_playwright.py 60 20 185
gptme/util/cli.py 51 31 174
gptme/tools/python.py 56 49 172
gptme/tools/save.py 32 38 167
gptme/util/__init__.py 37 22 158
gptme/prompts.py 58 83 157
gptme/tools/chats.py 43 49 142
gptme/tools/rag.py 61 75 141
gptme/llm/llm_openai_models.py 2 10 138
gptme/tools/__init__.py 41 17 138
tests/test_tool_use.py 8 18 127
gptme/config.py 36 14 123
tests/test_prompt.py 35 36 120
scripts/auto_rename_logs.py 34 18 115
scripts/gpt_todoer.py 47 25 107
scripts/train/collect.py 34 52 103
gptme/tools/subagent.py 37 29 102
gptme/tools/_browser_thread.py 20 3 97
scripts/treeofthoughts.py 33 35 95
tests/test_util_cli.py 32 23 94
gptme/util/export.py 17 25 86
tests/test_server.py 28 10 84
tests/test_shell.py 27 47 82
tests/test_tools.py 48 6 82
tests/test_prompt_tools.py 5 4 78
gptme/init.py 25 19 76
gptme/util/reduce.py 24 32 74
gptme/util/generate_name.py 5 6 69
gptme/codeblock.py 13 10 64
gptme/tools/browser.py 31 63 64
tests/test_tools_tts.py 29 37 60
tests/conftest.py 20 12 58
tests/test_context.py 18 13 58
gptme/eval/agents.py 12 5 56
gptme/dirs.py 27 14 55
gptme/eval/execenv.py 10 14 55
gptme/eval/suites/basic.py 15 1 55
gptme/eval/suites/init_projects.py 21 1 55
gptme/server/cli.py 8 5 55
scripts/describe_api.py 28 25 54
tests/test_tools_patch.py 36 91 54
gptme/tools/vision.py 19 12 53
tests/test_eval.py 8 9 49
gptme/__version__.py 9 7 45
gptme/util/cost.py 12 20 42
tests/test_message.py 7 17 39
tests/test_reduce.py 10 2 39
tests/test_codeblock.py 14 40 37
tests/test_util.py 28 51 37
gptme/util/clipboard.py 8 2 35
tests/test_tools_rag.py 14 7 35
gptme/tools/screenshot.py 11 9 34
gptme/util/terminal.py 23 21 34
tests/test_chat.py 17 28 34
scripts/shorten_details.py 11 6 33
gptme/eval/filestore.py 5 2 32
gptme/eval/types.py 13 12 32
scripts/list_user_messages.py 10 9 32
gptme/wut.py 13 12 31
gptme/tools/_browser_lynx.py 9 13 30
gptme/tools/youtube.py 10 1 26
scripts/summarize_project.py 11 1 26
tests/test_tools_python.py 19 1 26
gptme/util/useredit.py 12 13 21
gptme/util/interrupt.py 13 14 20
gptme/eval/suites/__init__.py 3 0 19
tests/test_browser.py 14 14 18
tests/test_logmanager.py 8 5 18
gptme/eval/suites/browser.py 5 0 16
gptme/tools/gh.py 14 21 16
tests/test_prompts.py 8 2 16
gptme/constants.py 7 11 13
gptme/tools/read.py 6 8 13
tests/test_tools_chats.py 2 0 13
tests/test_tools_shell.py 9 11 13
tests/test_tools_subagent.py 7 10 11
gptme/__init__.py 1 0 8
tests/test_config.py 2 0 5
gptme/eval/__init__.py 1 0 4
gptme/__main__.py 1 0 3
gptme/eval/__main__.py 1 0 3
gptme/server/__init__.py 2 3 3
gptme/server/__main__.py 1 0 3
-------------------------------------------------------------------------------------
SUM: 3143 2892 12042
-------------------------------------------------------------------------------------
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
Code Metrics#
$ make metrics
make[2]: Entering directory '/home/runner/work/gptme/gptme'
=== Code Metrics Summary ===
Project Overview:
Files: 109
Total blocks: 750
Average complexity: 4.009333333333333
Most Complex Functions (D+):
gptme/chat.py
F 48:0 chat - E
F 428:0 _parse_prompt - D
F 226:0 step - D
gptme/eval/main.py
F 199:0 main - D
gptme/cli.py
F 139:0 main - D
gptme/commands.py
F 82:0 handle_cmd - E
gptme/util/ask_execute.py
F 85:0 ask_execute - D
F 187:0 execute_with_confirmation - D
gptme/tools/tmux.py
F 160:0 execute_tmux - D
gptme/tools/computer.py
F 352:0 computer - D
scripts/auto_rename_logs.py
F 31:0 auto_rename_logs - D
Largest Files (>300 SLOC):
393 gptme/llm/llm_openai.py
387 gptme/tools/computer.py
387 gptme/chat.py
362 gptme/llm/llm_anthropic.py
361 gptme/eval/main.py
358 gptme/tools/base.py
356 gptme/tools/shell.py
329 gptme/logmanager.py
316 gptme/tools/tts.py
307 tests/test_cli.py
302 gptme/cli.py
301 gptme/util/prompt.py
make[2]: Leaving directory '/home/runner/work/gptme/gptme'
The metrics above show:
Project Overview: Basic stats about the codebase size and complexity
Complex Functions: Functions rated D+ (high complexity, should be refactored)
Large Files: Files over 300 SLOC (should be split into smaller modules)
We should aim to:
Keep average complexity below 4.0
Have no E-rated functions (extremely complex)
Have few D-rated functions (very complex)
Keep files under 300 SLOC where possible