Pre/beta - Unit Tests

[codebeaver-ai]

CodeBeaver Report

I started working from Pre/beta

🔄 2 test files added.
🐛 Found 1 bug
🛠️ 108/156 tests passed

🔄 Test Updates

I've added 2 tests. They all pass ☑️
New Tests:

• tests/test_chromium.py
• tests/test_scrape_do.py

No existing tests required updates.

🐛 Bug Detection

Potential issues:

• scrapegraphai/graphs/abstract_graph.py
  The error is occurring in the _create_llm method of the AbstractGraph class. Specifically, it's failing when trying to create a Bedrock model instance. The error message indicates that it's trying to pop a 'temperature' key from the llm_params dictionary, but this key doesn't exist.
  This suggests that the test is expecting the Bedrock model configuration to include a 'temperature' parameter, which is not being provided in the test case.
  The issue is not with the test itself, but with how the _create_llm method is handling the Bedrock model configuration. It's assuming that all Bedrock models will have a 'temperature' parameter, which may not always be the case.
  To fix this, the code should be modified to handle cases where the 'temperature' parameter is not provided for Bedrock models. This could be done by using the get method with a default value, or by checking if the key exists before trying to pop it.
  For example, the code could be changed to:

if llm_params["model_provider"] == "bedrock":
    llm_params["model_kwargs"] = {
        "temperature": llm_params.pop("temperature", None)  # Use None as default if not provided
    }

This change would allow the code to work correctly even when the 'temperature' parameter is not provided in the test configuration.

Test Error Log

tests.graphs.abstract_graph_test.TestAbstractGraph#test_create_llm[llm_config5-ChatBedrock]: self = <abstract_graph_test.TestGraph object at 0x7fa2b6a70d90>
llm_config = {'model': 'bedrock/anthropic.claude-3-sonnet-20240229-v1:0', 'region_name': 'IDK'}
    def _create_llm(self, llm_config: dict) -> object:
        """
        Create a large language model instance based on the configuration provided.
    
        Args:
            llm_config (dict): Configuration parameters for the language model.
    
        Returns:
            object: An instance of the language model client.
    
        Raises:
            KeyError: If the model is not supported.
        """
    
        llm_defaults = {"streaming": False}
        llm_params = {**llm_defaults, **llm_config}
        rate_limit_params = llm_params.pop("rate_limit", {})
    
        if rate_limit_params:
            requests_per_second = rate_limit_params.get("requests_per_second")
            max_retries = rate_limit_params.get("max_retries")
            if requests_per_second is not None:
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    llm_params["rate_limiter"] = InMemoryRateLimiter(
                        requests_per_second=requests_per_second
                    )
            if max_retries is not None:
                llm_params["max_retries"] = max_retries
    
        if "model_instance" in llm_params:
            try:
                self.model_token = llm_params["model_tokens"]
            except KeyError as exc:
                raise KeyError("model_tokens not specified") from exc
            return llm_params["model_instance"]
    
        known_providers = {
            "openai",
            "azure_openai",
            "google_genai",
            "google_vertexai",
            "ollama",
            "oneapi",
            "nvidia",
            "groq",
            "anthropic",
            "bedrock",
            "mistralai",
            "hugging_face",
            "deepseek",
            "ernie",
            "fireworks",
            "clod",
            "togetherai",
        }
    
        if "/" in llm_params["model"]:
            split_model_provider = llm_params["model"].split("/", 1)
            llm_params["model_provider"] = split_model_provider[0]
            llm_params["model"] = split_model_provider[1]
        else:
            possible_providers = [
                provider
                for provider, models_d in models_tokens.items()
                if llm_params["model"] in models_d
            ]
            if len(possible_providers) <= 0:
                raise ValueError(
                    f"""Provider {llm_params["model_provider"]} is not supported.
                                If possible, try to use a model instance instead."""
                )
            llm_params["model_provider"] = possible_providers[0]
            print(
                (
                    f"Found providers {possible_providers} for model {llm_params['model']}, using {llm_params['model_provider']}.\n"
                    "If it was not intended please specify the model provider in the graph configuration"
                )
            )
    
        if llm_params["model_provider"] not in known_providers:
            raise ValueError(
                f"""Provider {llm_params["model_provider"]} is not supported.
                             If possible, try to use a model instance instead."""
            )
    
        if llm_params.get("model_tokens", None) is None:
            try:
                self.model_token = models_tokens[llm_params["model_provider"]][
                    llm_params["model"]
                ]
            except KeyError:
                print(
                    f"""Max input tokens for model {llm_params["model_provider"]}/{llm_params["model"]} not found,
                    please specify the model_tokens parameter in the llm section of the graph configuration.
                    Using default token size: 8192"""
                )
                self.model_token = 8192
        else:
            self.model_token = llm_params["model_tokens"]
    
        try:
            if llm_params["model_provider"] not in {
                "oneapi",
                "nvidia",
                "ernie",
                "deepseek",
                "togetherai",
                "clod",
            }:
                if llm_params["model_provider"] == "bedrock":
                    llm_params["model_kwargs"] = {
>                       "temperature": llm_params.pop("temperature")
                    }
E                   KeyError: 'temperature'
scrapegraphai/graphs/abstract_graph.py:223: KeyError
During handling of the above exception, another exception occurred:
self = <abstract_graph_test.TestAbstractGraph object at 0x7fa2b6be8210>
llm_config = {'model': 'bedrock/anthropic.claude-3-sonnet-20240229-v1:0', 'region_name': 'IDK'}
expected_model = <class 'langchain_aws.chat_models.bedrock.ChatBedrock'>
    @pytest.mark.parametrize(
        "llm_config, expected_model",
        [
            (
                {"model": "openai/gpt-3.5-turbo", "openai_api_key": "sk-randomtest001"},
                ChatOpenAI,
            ),
            (
                {
                    "model": "azure_openai/gpt-3.5-turbo",
                    "api_key": "random-api-key",
                    "api_version": "no version",
                    "azure_endpoint": "https://www.example.com/",
                },
                AzureChatOpenAI,
            ),
            ({"model": "ollama/llama2"}, ChatOllama),
            ({"model": "oneapi/qwen-turbo", "api_key": "oneapi-api-key"}, OneApi),
            (
                {"model": "deepseek/deepseek-coder", "api_key": "deepseek-api-key"},
                DeepSeek,
            ),
            (
                {
                    "model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
                    "region_name": "IDK",
                },
                ChatBedrock,
            ),
        ],
    )
    def test_create_llm(self, llm_config, expected_model):
>       graph = TestGraph("Test prompt", {"llm": llm_config})
tests/graphs/abstract_graph_test.py:87: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/graphs/abstract_graph_test.py:19: in __init__
    super().__init__(prompt, config)
scrapegraphai/graphs/abstract_graph.py:60: in __init__
    self.llm_model = self._create_llm(config["llm"])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <abstract_graph_test.TestGraph object at 0x7fa2b6a70d90>
llm_config = {'model': 'bedrock/anthropic.claude-3-sonnet-20240229-v1:0', 'region_name': 'IDK'}
    def _create_llm(self, llm_config: dict) -> object:
        """
        Create a large language model instance based on the configuration provided.
    
        Args:
            llm_config (dict): Configuration parameters for the language model.
    
        Returns:
            object: An instance of the language model client.
    
        Raises:
            KeyError: If the model is not supported.
        """
    
        llm_defaults = {"streaming": False}
        llm_params = {**llm_defaults, **llm_config}
        rate_limit_params = llm_params.pop("rate_limit", {})
    
        if rate_limit_params:
            requests_per_second = rate_limit_params.get("requests_per_second")
            max_retries = rate_limit_params.get("max_retries")
            if requests_per_second is not None:
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    llm_params["rate_limiter"] = InMemoryRateLimiter(
                        requests_per_second=requests_per_second
                    )
            if max_retries is not None:
                llm_params["max_retries"] = max_retries
    
        if "model_instance" in llm_params:
            try:
                self.model_token = llm_params["model_tokens"]
            except KeyError as exc:
                raise KeyError("model_tokens not specified") from exc
            return llm_params["model_instance"]
    
        known_providers = {
            "openai",
            "azure_openai",
            "google_genai",
            "google_vertexai",
            "ollama",
            "oneapi",
            "nvidia",
            "groq",
            "anthropic",
            "bedrock",
            "mistralai",
            "hugging_face",
            "deepseek",
            "ernie",
            "fireworks",
            "clod",
            "togetherai",
        }
    
        if "/" in llm_params["model"]:
            split_model_provider = llm_params["model"].split("/", 1)
            llm_params["model_provider"] = split_model_provider[0]
            llm_params["model"] = split_model_provider[1]
        else:
            possible_providers = [
                provider
                for provider, models_d in models_tokens.items()
                if llm_params["model"] in models_d
            ]
            if len(possible_providers) <= 0:
                raise ValueError(
                    f"""Provider {llm_params["model_provider"]} is not supported.
                                If possible, try to use a model instance instead."""
                )
            llm_params["model_provider"] = possible_providers[0]
            print(
                (
                    f"Found providers {possible_providers} for model {llm_params['model']}, using {llm_params['model_provider']}.\n"
                    "If it was not intended please specify the model provider in the graph configuration"
                )
            )
    
        if llm_params["model_provider"] not in known_providers:
            raise ValueError(
                f"""Provider {llm_params["model_provider"]} is not supported.
                             If possible, try to use a model instance instead."""
            )
    
        if llm_params.get("model_tokens", None) is None:
            try:
                self.model_token = models_tokens[llm_params["model_provider"]][
                    llm_params["model"]
                ]
            except KeyError:
                print(
                    f"""Max input tokens for model {llm_params["model_provider"]}/{llm_params["model"]} not found,
                    please specify the model_tokens parameter in the llm section of the graph configuration.
                    Using default token size: 8192"""
                )
                self.model_token = 8192
        else:
            self.model_token = llm_params["model_tokens"]
    
        try:
            if llm_params["model_provider"] not in {
                "oneapi",
                "nvidia",
                "ernie",
                "deepseek",
                "togetherai",
                "clod",
            }:
                if llm_params["model_provider"] == "bedrock":
                    llm_params["model_kwargs"] = {
                        "temperature": llm_params.pop("temperature")
                    }
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    return init_chat_model(**llm_params)
            else:
                model_provider = llm_params.pop("model_provider")
    
                if model_provider == "clod":
                    return CLoD(**llm_params)
    
                if model_provider == "deepseek":
                    return DeepSeek(**llm_params)
    
                if model_provider == "ernie":
                    from langchain_community.chat_models import ErnieBotChat
    
                    return ErnieBotChat(**llm_params)
    
                elif model_provider == "oneapi":
                    return OneApi(**llm_params)
    
                elif model_provider == "togetherai":
                    try:
                        from langchain_together import ChatTogether
                    except ImportError:
                        raise ImportError(
                            """The langchain_together module is not installed.
                                          Please install it using 'pip install langchain-together'."""
                        )
                    return ChatTogether(**llm_params)
    
                elif model_provider == "nvidia":
                    try:
                        from langchain_nvidia_ai_endpoints import ChatNVIDIA
                    except ImportError:
                        raise ImportError(
                            """The langchain_nvidia_ai_endpoints module is not installed.
                                          Please install it using 'pip install langchain-nvidia-ai-endpoints'."""
                        )
                    return ChatNVIDIA(**llm_params)
    
        except Exception as e:
>           raise Exception(f"Error instancing model: {e}")
E           Exception: Error instancing model: 'temperature'
scrapegraphai/graphs/abstract_graph.py:266: Exception
tests.graphs.abstract_graph_test.TestAbstractGraph#test_create_llm_with_rate_limit[llm_config5-ChatBedrock]: self = <abstract_graph_test.TestGraph object at 0x7fa2b6a27810>
llm_config = {'model': 'bedrock/anthropic.claude-3-sonnet-20240229-v1:0', 'rate_limit': {'requests_per_second': 1}, 'region_name': 'IDK'}
    def _create_llm(self, llm_config: dict) -> object:
        """
        Create a large language model instance based on the configuration provided.
    
        Args:
            llm_config (dict): Configuration parameters for the language model.
    
        Returns:
            object: An instance of the language model client.
    
        Raises:
            KeyError: If the model is not supported.
        """
    
        llm_defaults = {"streaming": False}
        llm_params = {**llm_defaults, **llm_config}
        rate_limit_params = llm_params.pop("rate_limit", {})
    
        if rate_limit_params:
            requests_per_second = rate_limit_params.get("requests_per_second")
            max_retries = rate_limit_params.get("max_retries")
            if requests_per_second is not None:
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    llm_params["rate_limiter"] = InMemoryRateLimiter(
                        requests_per_second=requests_per_second
                    )
            if max_retries is not None:
                llm_params["max_retries"] = max_retries
    
        if "model_instance" in llm_params:
            try:
                self.model_token = llm_params["model_tokens"]
            except KeyError as exc:
                raise KeyError("model_tokens not specified") from exc
            return llm_params["model_instance"]
    
        known_providers = {
            "openai",
            "azure_openai",
            "google_genai",
            "google_vertexai",
            "ollama",
            "oneapi",
            "nvidia",
            "groq",
            "anthropic",
            "bedrock",
            "mistralai",
            "hugging_face",
            "deepseek",
            "ernie",
            "fireworks",
            "clod",
            "togetherai",
        }
    
        if "/" in llm_params["model"]:
            split_model_provider = llm_params["model"].split("/", 1)
            llm_params["model_provider"] = split_model_provider[0]
            llm_params["model"] = split_model_provider[1]
        else:
            possible_providers = [
                provider
                for provider, models_d in models_tokens.items()
                if llm_params["model"] in models_d
            ]
            if len(possible_providers) <= 0:
                raise ValueError(
                    f"""Provider {llm_params["model_provider"]} is not supported.
                                If possible, try to use a model instance instead."""
                )
            llm_params["model_provider"] = possible_providers[0]
            print(
                (
                    f"Found providers {possible_providers} for model {llm_params['model']}, using {llm_params['model_provider']}.\n"
                    "If it was not intended please specify the model provider in the graph configuration"
                )
            )
    
        if llm_params["model_provider"] not in known_providers:
            raise ValueError(
                f"""Provider {llm_params["model_provider"]} is not supported.
                             If possible, try to use a model instance instead."""
            )
    
        if llm_params.get("model_tokens", None) is None:
            try:
                self.model_token = models_tokens[llm_params["model_provider"]][
                    llm_params["model"]
                ]
            except KeyError:
                print(
                    f"""Max input tokens for model {llm_params["model_provider"]}/{llm_params["model"]} not found,
                    please specify the model_tokens parameter in the llm section of the graph configuration.
                    Using default token size: 8192"""
                )
                self.model_token = 8192
        else:
            self.model_token = llm_params["model_tokens"]
    
        try:
            if llm_params["model_provider"] not in {
                "oneapi",
                "nvidia",
                "ernie",
                "deepseek",
                "togetherai",
                "clod",
            }:
                if llm_params["model_provider"] == "bedrock":
                    llm_params["model_kwargs"] = {
>                       "temperature": llm_params.pop("temperature")
                    }
E                   KeyError: 'temperature'
scrapegraphai/graphs/abstract_graph.py:223: KeyError
During handling of the above exception, another exception occurred:
self = <abstract_graph_test.TestAbstractGraph object at 0x7fa2b6bea3d0>
llm_config = {'model': 'bedrock/anthropic.claude-3-sonnet-20240229-v1:0', 'rate_limit': {'requests_per_second': 1}, 'region_name': 'IDK'}
expected_model = <class 'langchain_aws.chat_models.bedrock.ChatBedrock'>
    @pytest.mark.parametrize(
        "llm_config, expected_model",
        [
            (
                {
                    "model": "openai/gpt-3.5-turbo",
                    "openai_api_key": "sk-randomtest001",
                    "rate_limit": {"requests_per_second": 1},
                },
                ChatOpenAI,
            ),
            (
                {
                    "model": "azure_openai/gpt-3.5-turbo",
                    "api_key": "random-api-key",
                    "api_version": "no version",
                    "azure_endpoint": "https://www.example.com/",
                    "rate_limit": {"requests_per_second": 1},
                },
                AzureChatOpenAI,
            ),
            (
                {"model": "ollama/llama2", "rate_limit": {"requests_per_second": 1}},
                ChatOllama,
            ),
            (
                {
                    "model": "oneapi/qwen-turbo",
                    "api_key": "oneapi-api-key",
                    "rate_limit": {"requests_per_second": 1},
                },
                OneApi,
            ),
            (
                {
                    "model": "deepseek/deepseek-coder",
                    "api_key": "deepseek-api-key",
                    "rate_limit": {"requests_per_second": 1},
                },
                DeepSeek,
            ),
            (
                {
                    "model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
                    "region_name": "IDK",
                    "rate_limit": {"requests_per_second": 1},
                },
                ChatBedrock,
            ),
        ],
    )
    def test_create_llm_with_rate_limit(self, llm_config, expected_model):
>       graph = TestGraph("Test prompt", {"llm": llm_config})
tests/graphs/abstract_graph_test.py:146: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/graphs/abstract_graph_test.py:19: in __init__
    super().__init__(prompt, config)
scrapegraphai/graphs/abstract_graph.py:60: in __init__
    self.llm_model = self._create_llm(config["llm"])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <abstract_graph_test.TestGraph object at 0x7fa2b6a27810>
llm_config = {'model': 'bedrock/anthropic.claude-3-sonnet-20240229-v1:0', 'rate_limit': {'requests_per_second': 1}, 'region_name': 'IDK'}
    def _create_llm(self, llm_config: dict) -> object:
        """
        Create a large language model instance based on the configuration provided.
    
        Args:
            llm_config (dict): Configuration parameters for the language model.
    
        Returns:
            object: An instance of the language model client.
    
        Raises:
            KeyError: If the model is not supported.
        """
    
        llm_defaults = {"streaming": False}
        llm_params = {**llm_defaults, **llm_config}
        rate_limit_params = llm_params.pop("rate_limit", {})
    
        if rate_limit_params:
            requests_per_second = rate_limit_params.get("requests_per_second")
            max_retries = rate_limit_params.get("max_retries")
            if requests_per_second is not None:
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    llm_params["rate_limiter"] = InMemoryRateLimiter(
                        requests_per_second=requests_per_second
                    )
            if max_retries is not None:
                llm_params["max_retries"] = max_retries
    
        if "model_instance" in llm_params:
            try:
                self.model_token = llm_params["model_tokens"]
            except KeyError as exc:
                raise KeyError("model_tokens not specified") from exc
            return llm_params["model_instance"]
    
        known_providers = {
            "openai",
            "azure_openai",
            "google_genai",
            "google_vertexai",
            "ollama",
            "oneapi",
            "nvidia",
            "groq",
            "anthropic",
            "bedrock",
            "mistralai",
            "hugging_face",
            "deepseek",
            "ernie",
            "fireworks",
            "clod",
            "togetherai",
        }
    
        if "/" in llm_params["model"]:
            split_model_provider = llm_params["model"].split("/", 1)
            llm_params["model_provider"] = split_model_provider[0]
            llm_params["model"] = split_model_provider[1]
        else:
            possible_providers = [
                provider
                for provider, models_d in models_tokens.items()
                if llm_params["model"] in models_d
            ]
            if len(possible_providers) <= 0:
                raise ValueError(
                    f"""Provider {llm_params["model_provider"]} is not supported.
                                If possible, try to use a model instance instead."""
                )
            llm_params["model_provider"] = possible_providers[0]
            print(
                (
                    f"Found providers {possible_providers} for model {llm_params['model']}, using {llm_params['model_provider']}.\n"
                    "If it was not intended please specify the model provider in the graph configuration"
                )
            )
    
        if llm_params["model_provider"] not in known_providers:
            raise ValueError(
                f"""Provider {llm_params["model_provider"]} is not supported.
                             If possible, try to use a model instance instead."""
            )
    
        if llm_params.get("model_tokens", None) is None:
            try:
                self.model_token = models_tokens[llm_params["model_provider"]][
                    llm_params["model"]
                ]
            except KeyError:
                print(
                    f"""Max input tokens for model {llm_params["model_provider"]}/{llm_params["model"]} not found,
                    please specify the model_tokens parameter in the llm section of the graph configuration.
                    Using default token size: 8192"""
                )
                self.model_token = 8192
        else:
            self.model_token = llm_params["model_tokens"]
    
        try:
            if llm_params["model_provider"] not in {
                "oneapi",
                "nvidia",
                "ernie",
                "deepseek",
                "togetherai",
                "clod",
            }:
                if llm_params["model_provider"] == "bedrock":
                    llm_params["model_kwargs"] = {
                        "temperature": llm_params.pop("temperature")
                    }
                with warnings.catch_warnings():
                    warnings.simplefilter("ignore")
                    return init_chat_model(**llm_params)
            else:
                model_provider = llm_params.pop("model_provider")
    
                if model_provider == "clod":
                    return CLoD(**llm_params)
    
                if model_provider == "deepseek":
                    return DeepSeek(**llm_params)
    
                if model_provider == "ernie":
                    from langchain_community.chat_models import ErnieBotChat
    
                    return ErnieBotChat(**llm_params)
    
                elif model_provider == "oneapi":
                    return OneApi(**llm_params)
    
                elif model_provider == "togetherai":
                    try:
                        from langchain_together import ChatTogether
                    except ImportError:
                        raise ImportError(
                            """The langchain_together module is not installed.
                                          Please install it using 'pip install langchain-together'."""
                        )
                    return ChatTogether(**llm_params)
    
                elif model_provider == "nvidia":
                    try:
                        from langchain_nvidia_ai_endpoints import ChatNVIDIA
                    except ImportError:
                        raise ImportError(
                            """The langchain_nvidia_ai_endpoints module is not installed.
                                          Please install it using 'pip install langchain-nvidia-ai-endpoints'."""
                        )
                    return ChatNVIDIA(**llm_params)
    
        except Exception as e:
>           raise Exception(f"Error instancing model: {e}")
E           Exception: Error instancing model: 'temperature'
scrapegraphai/graphs/abstract_graph.py:266: Exception

☂️ Coverage Improvements

Coverage improvements by file:

• tests/test_chromium.py

  New coverage: 17.24%
  Improvement: +0.00%

• tests/test_scrape_do.py

  New coverage: 100.00%
  Improvement: +29.41%

🎨 Final Touches

• I ran the hooks included in the pre-commit config.

---

_{Settings | Logs | CodeBeaver}

[github-actions]

🎉 This PR is included in version 1.47.0-beta.1 🎉

The release is available on:

• v1.47.0-beta.1
• GitHub release

Your semantic-release bot 📦🚀

[github-actions]

🎉 This PR is included in version 1.47.0 🎉

The release is available on:

• v1.47.0
• GitHub release

Your semantic-release bot 📦🚀