https://svtter.cn/en/tags/meter-reading/Recent content in Meter Reading on Svtter's BlogHugo -- gohugo.ioenThu, 19 Jun 2025 16:34:32 +0800https://svtter.cn/en/p/poor-performance-of-large-models-on-specific-tasks/Thu, 19 Jun 2025 16:34:32 +0800https://svtter.cn/en/p/poor-performance-of-large-models-on-specific-tasks/<img src="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/bg.png" alt="Featured image of post Poor Performance of Large Models on Specific Tasks" /><p>Vision large models perform poorly on some specific tasks but perform better with formatted text. Here, I use the localization of meter reading areas as an example to demonstrate the performance of large models.</p> <h2 id="source-code">Source Code </h2><p><a class="link" href="https://github.com/Svtter/vl-model/pull/4" target="_blank" rel="noopener" >https://github.com/Svtter/vl-model/pull/4</a></p> <h2 id="test-tasks">Test Tasks </h2><ol> <li>Extract text boxes from the image.</li> <li>Extract the meter reading area from the image.</li> </ol> <h2 id="test-file">Test File </h2><p><img src="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/meter-2.jpg" width="1280" height="1707" srcset="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/meter-2_hu_771f0f2490b85ed1.jpg 480w, https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/meter-2_hu_8848c0cff3902819.jpg 1024w" loading="lazy" alt="Original Meter" class="gallery-image" data-flex-grow="74" data-flex-basis="179px" ></p> <p>We can observe the performance differences among various models from these test results:</p> <h2 id="test-results-comparison">Test Results Comparison </h2><h3 id="results-using-bounding-boxes-as-prompts">Results Using Bounding Boxes as Prompts </h3><p><img src="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image.png" width="1280" height="1707" srcset="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_hu_41dee455ef817364.png 480w, https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_hu_4c0f4ae31905bed5.png 1024w" loading="lazy" alt="Overall Test Results" class="gallery-image" data-flex-grow="74" data-flex-basis="179px" ></p> <h3 id="detailed-performance-of-each-model">Detailed Performance of Each Model </h3><h4 id="anthropic-claude-35-sonnet">Anthropic Claude 3.5 Sonnet </h4><p><img src="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_anthropic_claude-3.5-sonnet.png" width="187" height="56" srcset="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_anthropic_claude-3.5-sonnet_hu_fef09b134291fdf1.png 480w, https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_anthropic_claude-3.5-sonnet_hu_d1e74a2cb60d339a.png 1024w" loading="lazy" alt="Claude 3.5 Sonnet Test Results" class="gallery-image" data-flex-grow="333" data-flex-basis="801px" ></p> <h4 id="google-gemini-25-pro">Google Gemini 2.5 Pro </h4><p><img src="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_google_gemini-2.5-pro.png" width="690" height="142" srcset="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_google_gemini-2.5-pro_hu_75fca6815db4fee4.png 480w, https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_google_gemini-2.5-pro_hu_50b7c46ce946b5fc.png 1024w" loading="lazy" alt="Gemini 2.5 Pro Test Results" class="gallery-image" data-flex-grow="485" data-flex-basis="1166px" ></p> <h4 id="openai-gpt-4o">OpenAI GPT-4o </h4><p><img src="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_openai_gpt-4o.png" width="120" height="60" srcset="https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_openai_gpt-4o_hu_e7a4998fc04bc3f0.png 480w, https://svtter.cn/p/poor-performance-of-large-models-on-specific-tasks/pics/cropped_image_openai_gpt-4o_hu_3305e7a6fcb0125a.png 1024w" loading="lazy" alt="GPT-4o Test Results" class="gallery-image" data-flex-grow="200" data-flex-basis="480px" ></p> <h2 id="analysis-summary">Analysis Summary </h2><p>From these test results, we can observe:</p> <ol> <li><strong>Differences in Visual Recognition Capabilities</strong>: Different models exhibit significant performance variations when handling the same visual task.</li> <li><strong>Formatted Text Processing</strong>: Compared to visual tasks, models perform more stably when processing structured text.</li> <li><strong>Model Characteristics</strong>: Each model has its unique strengths and limitations.</li> </ol> <p>These results remind us to evaluate the suitability of AI models based on specific task types when making selections.</p>