RFE: Lack of inventory module in OneView certified collection from HPE hardware provider.

[Alffernandez]

What are you experiencing? What are you expecting to happen?
Lack of inventory module in OneView certified HPE collection in Automation Hub. I expect to have visibility regarding this specific module.

As we can see, the collection hpe.oneview doesn't provide an inventory plugin/module in certified or community collection versions so, the customer is requesting a RFE in order to add this plugin into the collection.

[Torie-Coding]

Hi there, we also noticed some time ago that there is no inventory plugin available, so we built one ourselves. If it helps, I have attached it. The credentials are set as Special Credentials in AAP and as environment variables in the container.

How can customers request an RFE (Request for Enhancement)? Does this need to be submitted as a regular ticket for OneView?

# Author: Tobias Karger
# Date: 2023-11-30
# Description: Ansible inventory plugin for HPE OneView

import os
import requests
import warnings
import re
import sys
from ansible.plugins.inventory import BaseInventoryPlugin
from ansible.errors import AnsibleError
from urllib3.exceptions import InsecureRequestWarning

# Suppress InsecureRequestWarning if SSL verification is disabled
warnings.filterwarnings('ignore', category=InsecureRequestWarning)
DOCUMENTATION = '''
        name: hpe_oneview_inventory
        plugin_type: inventory
        short_description: HPE OneView inventory source
        description:
            - Fetches server hardware information from HPE OneView.
            - Requires credentials to authenticate with HPE OneView.
        extends_documentation_fragment:
            - inventory_cache
        options:
            plugin:
                description: The name of this plugin, it should always be set to hpe_oneview_inventory for this plugin to recognize it as it's own.
                required: true
                type: str
            oneview_hostname:
                description: Hostname of the HPE OneView appliance.
                required: true
                type: string
                default: false
            oneview_api_version:
                description: API version for HPE OneView.
                required: true
                type: string
            oneview_auth_domain:
                description: Authentication domain for HPE OneView.
                required: true
                type: string
            oneview_username:
                description: Username for HPE OneView.
                required: true
                type: string
            oneview_password:
                description: Password for HPE OneView.
                required: true
                type: string
            scope_uri:
                description: Optional URI for scope filtering. Include only the unique identifier of the scope.
                required: false
                type: string
            verify_ssl:
                description: Check if SSL certificate is valid.
                required: false
                type: bool
    '''

class InventoryModule(BaseInventoryPlugin):
    NAME = 'hpe_oneview_inventory'

    def __init__(self):
        super(InventoryModule, self).__init__()

    def verify_file(self, path):
        valid = super(InventoryModule, self).verify_file(path)
        return valid and (path.endswith('.yml') or path.endswith('.yaml'))

    def authenticate(self, hostname, api_version, domain, username, password, verify_ssl):
        print("Authenticating with HPE OneView...")
        url = f"https://{hostname}/rest/login-sessions"
        headers = {
            'X-Api-Version': api_version,
            'Content-Type': 'application/json'
        }
        payload = {
            'authLoginDomain': domain,
            'password': password,
            'userName': username,
            'loginMsgAck': True
        }
        response = requests.post(url, json=payload, headers=headers, verify=verify_ssl)
        if response.status_code != 200:
            print(f"Failed to authenticate. Status Code: {response.status_code}")
            raise AnsibleError("Error authenticating with HPE OneView")
        token = response.json().get('sessionID')
        print(f"Authentication successful. Token: {token}")
        return token

    def fetch_server_hardware(self, hostname, api_version, token, scope_uri=None, verify_ssl=True):
        print("Fetching server hardware from HPE OneView...")
        # Constructing the request URL
        if scope_uri:
            url = f"https://{hostname}/rest/server-hardware?count=1000&scopeUris=\"{scope_uri}\""
        else:
            url = f"https://{hostname}/rest/server-hardware?count=1000"

        headers = {
            'Auth': token,
            'X-Api-Version': api_version
        }

        # Fetching server hardware
        response = requests.get(url, headers=headers, verify=verify_ssl)
        if response.status_code != 200:
            print(f"Failed to fetch server hardware. Status Code: {response.status_code}")
            raise AnsibleError("Error fetching server hardware information from HPE OneView")

        data = response.json()
        servers = data.get('members', [])

        print(f"Number of servers fetched: {len(servers)}")

        # Process each server and extract required information
        server_info = self.process_servers(servers)
        print(f"Size of server_info element: {sys.getsizeof(server_info)}")
        print(f"Fetched and processed information of {len(servers)} servers.")

        print("Completed fetching all server hardware.")
        return server_info  # Return the collected server information

    def process_servers(self, servers):
        processed_server_info = []
        for server in servers:
            try: # Pre Gen10 has no portMap information. Some ILO´s are not reachable so no portMap ether.
                processed_info = {
                    'name': re.sub(r'^(.*),\s*bay\s*(\d+)$', r'\1_bay_\2', server.get('name')),
                    'OneView_hardware_name': server.get('name'),
                    'serial_number': server.get('serialNumber'),
                    'ip_address': next((addr['address'] for addr in server.get('mpHostInfo', {}).get('mpIpAddresses', []) if '.' in addr['address']), None),
                    'ilo_type': server.get('mpModel'),
                    'short_model': server.get('shortModel'),
                    'generation': server.get('generation'),
                    'mp_firmware_version': self.extract_version(server.get('mpFirmwareVersion')),
                    'platform': server.get('platform'),
                    'power_state': server.get('powerState'),
                    'processor_type': server.get('processorType'),
                    'rom_version': self.extract_version(server.get('romVersion'), pattern=r"[A-Za-z]+\d+ v\d+\.\d+"),
                    'scopes_uri': server.get('scopesUri'),
                    'fc_cards': self.extract_cards(server, 'FibreChannel'),
                    'eth_cards': self.extract_cards(server, 'Ethernet')
                }
                print(f"processed Server: {server.get('name')}")
                processed_server_info.append(processed_info)
            except:
                processed_info = {
                    'name': re.sub(r'^(.*),\s*bay\s*(\d+)$', r'\1_bay_\2', server.get('name')),
                    'OneView_hardware_name': server.get('name'),
                    'serial_number': server.get('serialNumber'),
                    'ip_address': next((addr['address'] for addr in server.get('mpHostInfo', {}).get('mpIpAddresses', []) if '.' in addr['address']), None),
                    'ilo_type': server.get('mpModel'),
                    'short_model': server.get('shortModel'),
                    'generation': server.get('generation'),
                    'mp_firmware_version': self.extract_version(server.get('mpFirmwareVersion')),
                    'platform': server.get('platform'),
                    'power_state': server.get('powerState'),
                    'processor_type': server.get('processorType'),
                    'rom_version': self.extract_version(server.get('romVersion'), pattern=r"[A-Za-z]+\d+ v\d+\.\d+"),
                    'scopes_uri': server.get('scopesUri'),
                }
                print(f"processed Server: {server.get('name')}")
                processed_server_info.append(processed_info)
        return processed_server_info

    def extract_version(self, version_string, pattern=r"\d+\.\d+"):
        version_pattern = re.compile(pattern)
        version_match = version_pattern.search(version_string)
        return version_match.group() if version_match else "Unknown"

    def extract_cards(self, server, card_type):
        cards = []
        for device_slot in server.get('portMap', {}).get('deviceSlots', []):
            device_name = device_slot.get('deviceName')
            device_number = device_slot.get('slotNumber')
            ports = []
            for physical_port in device_slot.get('physicalPorts', []):
                if physical_port.get('type') == card_type:
                    port_info = {
                        'port_number': physical_port.get('portNumber'),
                        'identifier': physical_port.get('wwn' if card_type == 'FibreChannel' else 'mac')
                    }
                    ports.append(port_info)
            if ports:
                card_info = {
                    'device_name': device_name,
                    'ports': ports,
                    'pcie_slot': device_number
                }
                cards.append(card_info)
        return cards

    def logout(self, hostname, api_version, token, verify_ssl=True):
        print("Logging out of HPE OneView...")
        url = f"https://{hostname}/rest/login-sessions"
        headers = {
            'X-Api-Version': api_version,
            'Auth': token
        }
        response = requests.delete(url, headers=headers, verify=verify_ssl)
        if response.status_code != 204:
            print(f"Failed to logout. Status Code: {response.status_code}")
            raise AnsibleError("Error logging out of HPE OneView")
        print("Logout successful.")

    def parse(self, inventory, loader, path, cache=True):
        super(InventoryModule, self).parse(inventory, loader, path, cache)

        # Read environment variables
        hostname = os.environ.get('oneview_hostname')
        api_version = os.environ.get('oneview_api_version')
        domain = os.environ.get('oneview_auth_domain')
        username = os.environ.get('oneview_username')
        password = os.environ.get('oneview_password')
        scope_uri = os.environ.get('scope_uri')
        verify_ssl = os.environ.get('verify_ssl', 'True').lower() in ['true', '1', 'yes']

        # Debug: Print out loaded variables
        print(f"Hostname: {hostname}")
        print(f"API Version: {api_version}")
        print(f"Domain: {domain}")
        print(f"Username: {username}")
        print(f"Password: {'******' if password else None}")
        print(f"Scope URI: {scope_uri}")
        print(f"Verify SSL: {verify_ssl}")

        # Check if all required variables are present
        required_vars = [hostname, api_version, domain, username, password]
        missing_vars = [var for var, val in zip(['hostname', 'api_version', 'domain', 'username', 'password'], required_vars) if not val]
        if missing_vars:
            raise AnsibleError(f"Missing required environment variables: {', '.join(missing_vars)}")

        # Suppress SSL warnings if verify_ssl is false
        if not verify_ssl:
            warnings.filterwarnings('ignore', category=InsecureRequestWarning)

        token = self.authenticate(hostname, api_version, domain, username, password, verify_ssl)
        try:
            server_data = self.fetch_server_hardware(hostname, api_version, token, scope_uri, verify_ssl)
            print(f"Total servers processed: {len(server_data)}")
            group_name = hostname
            self.inventory.add_group(group_name)
            for server in server_data:
                server_name = server.get('name')
                if server_name:
                    self.inventory.add_host(server_name, group=group_name)
                    self.inventory.set_variable(server_name, 'serial_number', server.get('serial_number'))
                    self.inventory.set_variable(server_name, 'OneView_hardware_name', server.get('OneView_hardware_name'))
                    self.inventory.set_variable(server_name, 'ansible_host', server.get('ip_address'))
                    self.inventory.set_variable(server_name, 'ILO_type', server.get('ilo_type'))
                    self.inventory.set_variable(server_name, 'shortModel', server.get('short_model'))
                    self.inventory.set_variable(server_name, 'generation', server.get('generation'))
                    self.inventory.set_variable(server_name, 'ILO_firmware', server.get('mp_firmware_version'))
                    self.inventory.set_variable(server_name, 'rom_version', server.get('rom_version'))
                    self.inventory.set_variable(server_name, 'platform', server.get('platform'))
                    self.inventory.set_variable(server_name, 'powerState', server.get('power_state'))
                    self.inventory.set_variable(server_name, 'processorType', server.get('processor_type'))
                    self.inventory.set_variable(server_name, 'fibre_channel_cards', server.get('fc_cards'))
                    self.inventory.set_variable(server_name, 'eth_cards', server.get('eth_cards'))
                    self.inventory.set_variable(server_name, 'scopesUri', server.get('scopes_uri'))
            print("All servers have been added to the inventory.")

        finally:
            self.logout(hostname, api_version, token, verify_ssl)

[akshith-gunasheelan]

Hi @Torie-Coding, we will check this and get back to you.

[akshith-gunasheelan]

After going through your code, we see that the parameters which you are gathering for each server under inventory is already present under the role oneview_server_hardware_facts. https://github.com/HewlettPackard/oneview-ansible-collection/blob/master/roles/oneview_server_hardware_facts/tasks/main.yml

Please let us know if there is anything else we can help with.

[Torie-Coding]

Hello @akshith-gunasheelan,

Yes, what I'm doing there is similar to the role oneview_server_hardware_facts. However, this issue was about the need for an inventory plugin. The script above is our current inventory script that we wrote and are using ourselves. I shared it with you in case it helps with the implementation.

Or did I misunderstand, and you mean that a role can be used as an inventory plugin for Ansible Tower/AAP?

Additionally, I had a question about how we, as HPE customers, can submit an RFE (Request for Enhancement) to you. Does this happen within the framework of a case, or through the ASM?

[akshith-gunasheelan]

Thanks for your script. Looks like this is a very specific ask from your side. The oneview_server_hardware_facts provides all the data available from OneView.
Regarding your question if a role can be used as an inventory plugin for Ansible Tower/ AAP, we have not validated it on these platforms. Will put this in our backlog and take it up as per our priority.
For any request regarding RFE please contact - [email protected] with the following details -
Requirement, Severity, Impact for customer, Any work around, Customer Information (which server model they use, how many etc.)