CG-EXP-1: GENI AM API Compliance
GENI AM API v2 tests for NCSU2 and NCSU Cisco racks.
Test Status
This section captures the status for each step in the acceptance test plan.
Test | State | Tickets/Comments |
Omni Commands | Advertisement problem: unknown component id; | |
NCSU2 Compliance | Advertisement problem: NCSU2-only no VLAN failure even though resources are available? | |
NCSU Compliance | Advertisement problem: 2)NCSU-only unkown hardware type |
State Legend | Description |
Color(green,Pass)? | Test completed and met all criteria |
Color(#98FB98,Pass: most criteria)? | Test completed and met most criteria. Exceptions documented |
Color(red,Fail)? | Test completed and failed to meet criteria. |
Color(yellow,Complete)? | Test completed but will require re-execution due to expected changes |
Color(orange,Blocked)? | Blocked by ticketed issue(s). |
Color(#63B8FF,In Progress)? | Currently under test. |
Omni Commands
Collect getversion information to verify that NCSU is running the same Orca version as NCSU2.
$ omni getversion -a ncsu-eg -o ; omni getversion -a ncsu2-eg -o $ egrep orca_version getversion-exogeni-net-ncsu* getversion-exogeni-net-ncsu2vmsite.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-6427'} getversion-exogeni-net-ncsuvmsite.json: 'orca_version': 'ORCA Dungeness: v.4.0-SNAPSHOT.build-6427'}
Both racks are running the same version.
Verify compute resources available in NCSU2 rack:
$ omni listresources -a ncsu2-eg -o; egrep "node c" rspec-exogeni-net-ncsu2vmsite.xml <node component_id="urn:publicid:IDN+exogeni.net:ncsu2vmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:ncsu2vmsite+authority+am" component_name="orca-vm-cloud" exclusive="false"> <node component_id="urn:publicid:IDN+exogeni.net:ncsu2Net+node+orca-transit-net-vlan" component_manager_id="urn:publicid:IDN+exogeni.net:ncsu2Net+authority+am" component_name="orca-transit-net-vlan" exclusive="false">
Verify compute resources available in NCSU rack:
$ omni listresources -a ncsu-eg -o; egrep "node c" rspec-exogeni-net-ncsuvmsite.xml <node component_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+node+unknown" component_manager_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am" component_name="unknown" exclusive="false"> <node component_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+node+orca-vm-cloud" component_manager_id="urn:publicid:IDN+exogeni.net:ncsuvmsite+authority+am" component_name="orca-vm-cloud" exclusive="false">
Note: What is "unknown" component_id for both NCSU and NCSU2?
Note: No Bare Metal nodes available
Running rspeclint on the Advertisement generated by listresources for NCSU2 passes:
$ rspeclint rspec-exogeni-net-ncsu2vmsite.xml
The rspeclint does not pass for NCSU:
$ rspeclint rspec-exogeni-net-ncsuvmsite.xml Line 7: Failed validation with root at element: Schemas validity error : Element '{http://www.geni.net/resources/rspec/3}hardware_type': The attribute 'name' is required but missing. : rspec
Note: The "<hardware_type/>" on line 10 should be removed from the advertisement.
Note: NCSU2 advertises stitching, NCSU does not... investigating.
CG-EXP-1a GENI AM API Compliance - NCSU2
This test verifies compliance for the NCSU2 Cisco rack.
NCSU2 Unbound Resources
The AM API Acceptance test results for NCSU2:
lnevers@sendaria:~/gcf-2.7/acceptance_tests/AM_API$ ./am_api_accept.py -a am-undertest --un-bound E.E.......... ====================================================================== ERROR: test_CreateSliver: Passes if the sliver creation workflow succeeds. ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 773, in test_CreateSliver self.subtest_CreateSliverWorkflow() File "./am_api_accept.py", line 803, in subtest_CreateSliverWorkflow numslivers, manifest, slivers = self.subtest_generic_CreateSliver( slicename, doProvision, doPOA, expectedExpiration=sliceExpiration ) File "./am_api_accept.py", line 1881, in subtest_generic_CreateSliver numslivers, manifest = self.subtest_CreateSliver( slicename ) File "./am_api_accept.py", line 1610, in subtest_CreateSliver text, manifest = self.call(omniargs, self.options_copy) File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call ret_val = omni.call( cmd, options=options, verbose=True ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call return API_call( framework, config, args, opts, verbose=verbose ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call result = handler._handle(args) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle (message, val) = getattr(self.amhandler,call)(args[1:]) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 1840, in createsliver (result, message) = self._retrieve_value(result, message, self.framework) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value self._raise_omni_error(message, AMAPIError, result) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error raise err, (msg, triple) AMAPIError: AMAPIError: Error from Aggregate: code 2: Embedding workflow ERROR: 1:No available resources in domain: http://geni-orca.renci.org/owl/ncsuvmsite.rdf#ncsuvmsite/Domain/vlan:vlan; Requested: vlan:1. Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.. ====================================================================== ERROR: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 1141, in test_CreateSliverWorkflow_multiSlice createReturn = self.subtest_generic_CreateSliver( slicenames[i], expectedExpiration=sliceExpiration ) File "./am_api_accept.py", line 1881, in subtest_generic_CreateSliver numslivers, manifest = self.subtest_CreateSliver( slicename ) File "./am_api_accept.py", line 1610, in subtest_CreateSliver text, manifest = self.call(omniargs, self.options_copy) File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call ret_val = omni.call( cmd, options=options, verbose=True ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call return API_call( framework, config, args, opts, verbose=verbose ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call result = handler._handle(args) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle (message, val) = getattr(self.amhandler,call)(args[1:]) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 1840, in createsliver (result, message) = self._retrieve_value(result, message, self.framework) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value self._raise_omni_error(message, AMAPIError, result) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error raise err, (msg, triple) AMAPIError: AMAPIError: Error from Aggregate: code 2: Embedding workflow ERROR: 1:No available resources in domain: http://geni-orca.renci.org/owl/ncsuvmsite.rdf#ncsuvmsite/Domain/vlan:vlan; Requested: vlan:1. Please see https://geni-orca.renci.org/trac/wiki/orca-errors for possible solutions.. ---------------------------------------------------------------------- Ran 13 tests in 210.520s FAILED (errors=2)
Note: Lack or VLAN resources is reported, but could not duplicate... investigating
NCSU2 Bound Resource
The results:
$ ./am_api_accept.py -a am-undertest -V2 F.F.......... ====================================================================== FAIL: test_CreateSliver: Passes if the sliver creation workflow succeeds. ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 773, in test_CreateSliver self.subtest_CreateSliverWorkflow() File "./am_api_accept.py", line 819, in subtest_CreateSliverWorkflow "Created sliver") File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 271, in assertManifestMatchesRequest str(msg)+": Request RSpec and Manifest RSpec " \ File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual raise NotEqualComponentIDsError, msg NotEqualComponentIDsError: Created sliver: Request RSpec and Manifest RSpec expected to have same component_ids but did not. ====================================================================== FAIL: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 1179, in test_CreateSliverWorkflow_multiSlice "but did not." % slicenames[i]) File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual raise NotEqualComponentIDsError, msg NotEqualComponentIDsError: Request RSpec and Manifest RSpec returned by 'ListResources' on slice 'acclne-1605420' expected to have same component_ids but did not. ---------------------------------------------------------------------- Ran 13 tests in 247.947s FAILED (failures=2)
Note: may be related to "unknown" component_id
CG-EXP-1b GENI AM API Compliance - NCSU
This test verifies compliance for the NCSU Cisco rack.
NCSU Unbound Resources
The AM API Acceptance test results:
$ ./am_api_accept.py -a am-undertest --un-bound E.E.......... ====================================================================== ERROR: test_CreateSliver: Passes if the sliver creation workflow succeeds. ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 773, in test_CreateSliver self.subtest_CreateSliverWorkflow() File "./am_api_accept.py", line 929, in subtest_CreateSliverWorkflow self.subtest_generic_RenewSliver_many( slicename ) File "./am_api_accept.py", line 1933, in subtest_generic_RenewSliver_many self.subtest_RenewSliver_many( slicename ) File "./am_api_accept.py", line 1410, in subtest_RenewSliver_many self.subtest_RenewSliver( slicename, twodays ) File "./am_api_accept.py", line 1375, in subtest_RenewSliver text, (succList, failList) = self.call(omniargs, self.options_copy) File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call ret_val = omni.call( cmd, options=options, verbose=True ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call return API_call( framework, config, args, opts, verbose=verbose ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call result = handler._handle(args) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle (message, val) = getattr(self.amhandler,call)(args[1:]) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 2749, in renewsliver (res, message) = self._retrieve_value(res, message, self.framework) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value self._raise_omni_error(message, AMAPIError, result) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error raise err, (msg, triple) AMAPIError: AMAPIError: Error from Aggregate: code 1: renewSlice(): renewal term shorter than original slice end is not valid.. ====================================================================== ERROR: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 1267, in test_CreateSliverWorkflow_multiSlice self.subtest_generic_RenewSliver_many( slicenames[i] ) File "./am_api_accept.py", line 1933, in subtest_generic_RenewSliver_many self.subtest_RenewSliver_many( slicename ) File "./am_api_accept.py", line 1410, in subtest_RenewSliver_many self.subtest_RenewSliver( slicename, twodays ) File "./am_api_accept.py", line 1375, in subtest_RenewSliver text, (succList, failList) = self.call(omniargs, self.options_copy) File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 172, in call ret_val = omni.call( cmd, options=options, verbose=True ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 719, in call return API_call( framework, config, args, opts, verbose=verbose ) File "/home/lnevers/gcf-2.7/src/gcf/oscript.py", line 793, in API_call result = handler._handle(args) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/handler.py", line 90, in _handle (message, val) = getattr(self.amhandler,call)(args[1:]) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 2749, in renewsliver (res, message) = self._retrieve_value(res, message, self.framework) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5382, in _retrieve_value self._raise_omni_error(message, AMAPIError, result) File "/home/lnevers/gcf-2.7/src/gcf/omnilib/amhandler.py", line 5486, in _raise_omni_error raise err, (msg, triple) AMAPIError: AMAPIError: Error from Aggregate: code 1: renewSlice(): renewal term shorter than original slice end is not valid.. ---------------------------------------------------------------------- Ran 13 tests in 674.790s FAILED (errors=2)
Note: New issue... Expiration should not be causing issues....Investigating
NCSU Bound Compute Resource
The results:
$ ./am_api_accept.py -a am-undertest F.F.......... ====================================================================== FAIL: test_CreateSliver: Passes if the sliver creation workflow succeeds. ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 773, in test_CreateSliver self.subtest_CreateSliverWorkflow() File "./am_api_accept.py", line 819, in subtest_CreateSliverWorkflow "Created sliver") File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 271, in assertManifestMatchesRequest str(msg)+": Request RSpec and Manifest RSpec " \ File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual raise NotEqualComponentIDsError, msg NotEqualComponentIDsError: Created sliver: Request RSpec and Manifest RSpec expected to have same component_ids but did not. ====================================================================== FAIL: test_CreateSliverWorkflow_multiSlice: Do CreateSliver workflow with multiple slices ---------------------------------------------------------------------- Traceback (most recent call last): File "./am_api_accept.py", line 1179, in test_CreateSliverWorkflow_multiSlice "but did not." % slicenames[i]) File "/home/lnevers/gcf-2.7/src/omni_unittest.py", line 253, in assertCompIDsEqual raise NotEqualComponentIDsError, msg NotEqualComponentIDsError: Request RSpec and Manifest RSpec returned by 'ListResources' on slice 'acclne-1539130' expected to have same component_ids but did not. ---------------------------------------------------------------------- Ran 13 tests in 247.056s FAILED (failures=2)
Note: may be related to "unknown" component_id