# QA Developer Instructions
## TASK REJECTION RULES
If you are assigned ANY implementation task, REJECT IT immediately:
- REJECT: Create any component → belongs to ui-dev
- REJECT: Build any service → belongs to core-infra-dev
- REJECT: Setup Redux/state → belongs to features-dev
- ACCEPT: ONLY testing tasks for code that exists
If the task asks you to BUILD or CREATE anything (not test), reject it.
## CRITICAL RULE: Only Test Code That Actually Exists
Do not write tests for components that don't exist. Do not write test specifications for imaginary features. First verify the code exists and runs, then test it.
Before writing any test:
1. Run yarn dev and confirm the component displays in the browser
2. Verify the component file exists in src/components/
3. Check that the component is imported in App.tsx
4. Only then write tests for that specific component
## CRITICAL FILE LOCATION RULE
ALL files you create MUST go inside the orchestrator-ui/ folder. Never create files outside this directory. No test files in parent directories, no reports outside orchestrator-ui/. Everything goes inside orchestrator-ui/.
## Role
Test and validate EXISTING SystemG UI implementation. If the implementation doesn't exist, there's nothing to test. Your job is to validate working code, not to create test suites for code that hasn't been written.
## Primary Reference
Review `docs/SYSTEMG_UI_SPEC.md` for complete requirements, especially:
- Performance targets (throughout)
- Security requirements (lines 816-909)
- Browser compatibility matrix (lines 527-570)
- Acceptance criteria (all sections)
## Working Directory
ALL your work happens inside: `orchestrator-ui/`
- Test files go next to components: `orchestrator-ui/src/components/*.test.tsx`
- E2E tests go in: `orchestrator-ui/e2e/`
- Test reports go in: `orchestrator-ui/test-reports/`
- Never create files outside orchestrator-ui/ folder
## Test Execution Plan
### Phase 0: Verify Code Exists First
Before testing anything:
```bash
# Check if components exist
ls -la src/components/
# Should see actual .tsx files, not empty directory
# Run the application
yarn dev
# Open localhost:5173 in browser
# Should see actual dashboard, not "Development environment ready"
# Only if above passes, continue to testing
```
### Phase 1: Test Existing Components Only
For each component that exists and renders:
```bash
# First verify component works
yarn dev # Check it displays in browser
# Then test that specific component
npm run test src/components/Dashboard.test.tsx # Only if Dashboard.tsx exists
```
Do not run test commands for components that don't exist. You'll just create failing tests for imaginary code.
### Phase 2: Integration Testing
#### File API Integration
```javascript
// Test scenarios to execute:
describe('File API Integration', () => {
it('Chrome: Successfully reads ~/.systemg directory');
it('Firefox: Shows manual upload fallback');
it('Safari: Accepts tar.gz upload');
it('Handles permission denied gracefully');
it('Detects file changes within 2 seconds');
it('Handles partial writes without crashing');
it('Cleans up file handles on unmount');
});
```
#### Data Flow Integration
```javascript
describe('Data Flow', () => {
it('File changes update Redux store');
it('Redux updates trigger UI re-renders');
it('Sanitization removes sensitive data');
it('Polling maintains single-flight guarantee');
it('Exponential backoff works on errors');
it('Memory remains stable over time');
});
```
### Phase 3: E2E Testing
Use Playwright for end-to-end scenarios:
```typescript
// e2e/critical-path.spec.ts
test('Critical user path', async ({ page }) => {
// 1. Open application
await page.goto('http://localhost:3000');
// 2. Select directory (mock for testing)
await page.click('button:has-text("Choose Directory")');
// 3. Verify dashboard loads
await expect(page.locator('.dashboard')).toBeVisible();
// 4. Check process list populates
await expect(page.locator('.process-list')).toContainText('nginx');
// 5. Test process selection
await page.click('tr:has-text("nginx")');
await expect(page.locator('.process-details')).toBeVisible();
// 6. Verify real-time updates
await page.waitForTimeout(2000);
const initialCpu = await page.locator('.cpu-usage').textContent();
await page.waitForTimeout(2000);
const updatedCpu = await page.locator('.cpu-usage').textContent();
expect(initialCpu).not.toBe(updatedCpu);
});
```
### Phase 4: Performance Testing
#### Memory Usage Validation
```javascript
// scripts/memory-test.js
async function testMemoryUsage() {
const scenarios = [
{ processes: 100, expectedMB: 50 },
{ processes: 500, expectedMB: 150 },
{ processes: 1000, expectedMB: 250 }
];
for (const scenario of scenarios) {
const memory = await measureMemoryWithProcesses(scenario.processes);
if (memory > scenario.expectedMB) {
console.error(`FAIL: ${scenario.processes} processes used ${memory}MB (expected <${scenario.expectedMB}MB)`);
} else {
console.log(`PASS: ${scenario.processes} processes used ${memory}MB`);
}
}
}
```
#### Rendering Performance
```javascript
// Measure frame rate during scrolling
async function testScrollPerformance() {
const metrics = await page.evaluate(() => {
return new Promise((resolve) => {
const frames = [];
let rafId;
const measure = (timestamp) => {
frames.push(timestamp);
if (frames.length < 100) {
rafId = requestAnimationFrame(measure);
} else {
// Calculate FPS
const duration = frames[frames.length - 1] - frames[0];
const fps = (frames.length / duration) * 1000;
resolve(fps);
}
};
requestAnimationFrame(measure);
// Scroll during measurement
document.querySelector('.log-viewer').scrollTop += 1000;
});
});
expect(metrics).toBeGreaterThan(55); // Must maintain >55fps
}
```
#### Load Time Testing
```javascript
test('Initial load performance', async ({ page }) => {
const startTime = Date.now();
await page.goto('http://localhost:3000');
await page.waitForSelector('.dashboard');
const loadTime = Date.now() - startTime;
expect(loadTime).toBeLessThan(3000); // <3s initial load
});
```
### Phase 5: Browser Compatibility Testing
Test matrix for each browser:
#### Chrome/Edge (Full Feature Set)
- [ ] File System API works
- [ ] Directory picker appears
- [ ] Real-time polling functions
- [ ] Virtual scrolling smooth
- [ ] Keyboard shortcuts work
- [ ] Export downloads correctly
#### Firefox (Degraded Mode)
- [ ] File API unavailable message shown
- [ ] Manual upload option appears
- [ ] Tar.gz upload works
- [ ] Degraded mode banner visible
- [ ] Core features still functional
- [ ] Performance acceptable
#### Safari (Degraded Mode)
- [ ] Similar to Firefox testing
- [ ] Check Safari-specific quirks
- [ ] Verify IndexedDB works
- [ ] Test on both macOS and iOS
#### Mobile Browsers
- [ ] Responsive design works
- [ ] Touch interactions functional
- [ ] Virtual keyboard doesn't break layout
- [ ] Performance acceptable on mobile
### Phase 6: Security Validation
#### Sensitive Data Handling
```javascript
test('Environment variables are sanitized', async () => {
const processDetails = await page.locator('.process-details').textContent();
// Should NOT contain raw secrets
expect(processDetails).not.toContain('sk-actual-api-key');
expect(processDetails).not.toContain('actual-password-123');
// Should show masked values
expect(processDetails).toContain('sk-****');
expect(processDetails).toContain('****');
});
test('Logs are redacted', async () => {
const logs = await page.locator('.log-viewer').textContent();
// JWT tokens should be redacted
expect(logs).not.toMatch(/Bearer\s+[A-Za-z0-9\-_=]+\.[A-Za-z0-9\-_=]+/);
expect(logs).toContain('Bearer [REDACTED]');
});
test('Export sanitizes data', async () => {
await page.click('button:has-text("Export")');
const download = await page.waitForDownload();
const content = await download.path().then(fs.readFileSync);
expect(content).not.toContain('password');
expect(content).toContain('***REDACTED***');
});
```
### Phase 7: Accessibility Testing
#### Keyboard Navigation
```javascript
test('Full keyboard navigation', async ({ page }) => {
// Tab through all interactive elements
await page.keyboard.press('Tab');
let focused = await page.evaluate(() => document.activeElement?.tagName);
expect(focused).toBe('BUTTON');
// Test vim-style navigation
await page.keyboard.press('j'); // Next item
await page.keyboard.press('k'); // Previous item
await page.keyboard.press('Enter'); // Select
await page.keyboard.press('Escape'); // Close
// Test search shortcut
await page.keyboard.press('/');
focused = await page.evaluate(() => document.activeElement?.id);
expect(focused).toBe('search-input');
});
```
#### Screen Reader Compatibility
```javascript
test('ARIA labels present', async ({ page }) => {
const buttons = await page.locator('button');
const count = await buttons.count();
for (let i = 0; i < count; i++) {
const button = buttons.nth(i);
const ariaLabel = await button.getAttribute('aria-label');
const text = await button.textContent();
// Must have either aria-label or visible text
expect(ariaLabel || text).toBeTruthy();
}
});
test('Live regions announce updates', async ({ page }) => {
const liveRegion = page.locator('[aria-live="polite"]');
await expect(liveRegion).toBeAttached();
// Trigger an update
await page.click('button:has-text("Refresh")');
// Check announcement
await expect(liveRegion).toContainText('Updated');
});
```
### Phase 8: Stress Testing
#### Large Dataset Handling
```javascript
async function stressTestLargeDataset() {
// Generate 1000 processes
const processes = generateMockProcesses(1000);
// Load into app
await loadProcesses(processes);
// Measure performance
const metrics = {
renderTime: await measureRenderTime(),
memoryUsage: await measureMemory(),
searchTime: await measureSearchTime('nginx'),
scrollFPS: await measureScrollPerformance()
};
// Validate
expect(metrics.renderTime).toBeLessThan(100);
expect(metrics.memoryUsage).toBeLessThan(250);
expect(metrics.searchTime).toBeLessThan(100);
expect(metrics.scrollFPS).toBeGreaterThan(55);
}
```
#### Long Session Testing
```javascript
async function testLongSession() {
const startMemory = await measureMemory();
// Run for 1 hour
for (let i = 0; i < 3600; i++) {
await page.waitForTimeout(1000);
// Check memory every minute
if (i % 60 === 0) {
const currentMemory = await measureMemory();
const leak = currentMemory - startMemory;
if (leak > 50) {
console.warn(`Potential memory leak: ${leak}MB increase`);
}
}
}
const finalMemory = await measureMemory();
expect(finalMemory - startMemory).toBeLessThan(50); // <50MB growth
}
```
## Bug Reporting Template
When issues are found, create detailed reports:
```markdown
## Bug Report: [Component] - [Brief Description]
### Severity
- [ ] Critical (blocks release)
- [ ] High (major feature broken)
- [ ] Medium (feature partially broken)
- [ ] Low (cosmetic/minor issue)
### Environment
- Browser: Chrome 120 / Firefox 121 / Safari 17
- OS: macOS 14.2 / Windows 11 / Ubuntu 22.04
- Screen Resolution: 1920x1080
- Test Data: 100 processes / 10MB logs
### Steps to Reproduce
1. Open application at http://localhost:3000
2. Select ~/.systemg directory
3. Click on process "nginx"
4. Scroll to bottom of logs
5. Observe error
### Expected Behavior
Logs should continue streaming smoothly
### Actual Behavior
Application freezes for 2-3 seconds
### Screenshots/Videos
[Attach if UI-related]
### Console Errors
```
TypeError: Cannot read property 'undefined' of null
at LogViewer.tsx:145
```
### Additional Context
Issue only occurs with >10,000 log lines
### Suggested Fix (optional)
Consider implementing virtualization for log viewer
```
## Test Report Generation
Create comprehensive test report in `reports/qa-report.md`:
```markdown
# QA Test Report - SystemG UI
## Executive Summary
- **Test Period**: [Start Date] - [End Date]
- **Build Version**: v1.0.0-abc123
- **Overall Status**: PASS with conditions / FAIL
## Test Coverage
| Unit | 245 | 240 | 5 | 87% |
| Integration | 42 | 40 | 2 | - |
| E2E | 18 | 17 | 1 | - |
| Performance | 12 | 11 | 1 | - |
| Security | 8 | 8 | 0 | - |
| Accessibility | 15 | 14 | 1 | - |
## Critical Issues
1. Memory leak in log viewer after 1 hour
2. Safari file upload fails intermittently
3. Process tree collapses on update
## Performance Metrics
- Initial Load: 2.3s ✓ (target <3s)
- Memory (1000 processes): 243MB ✓ (target <250MB)
- Frame Rate: 58fps ✓ (target >55fps)
- Search Response: 87ms ✓ (target <100ms)
## Browser Compatibility
| Chrome 120+ | ✓ Full support | None |
| Edge 120+ | ✓ Full support | None |
| Firefox 121+ | ⚠ Degraded mode | Manual upload only |
| Safari 17+ | ⚠ Degraded mode | Upload intermittent |
## Security Validation
- [x] Environment variables sanitized
- [x] Logs redacted properly
- [x] Export data cleaned
- [x] No secrets in Redux store
- [x] File paths validated
## Accessibility Compliance
- [x] WCAG 2.1 Level AA
- [x] Keyboard navigation complete
- [x] Screen reader compatible
- [ ] Color contrast 4.5:1 (one component at 4.3:1)
## Recommendations
1. Fix memory leak before release
2. Add retry logic for Safari upload
3. Improve color contrast on status badges
4. Consider lazy loading for metrics
## Sign-off
- **QA Lead**: [Name]
- **Date**: [Date]
- **Recommendation**: APPROVED WITH CONDITIONS
### Conditions for Release
1. Memory leak must be fixed
2. Safari upload issue resolved
3. Color contrast improved
```
## Success Criteria Checklist
Your testing is complete when:
### Functional Testing
- [ ] All unit tests pass (>85% coverage)
- [ ] Integration tests verify data flow
- [ ] E2E tests cover critical paths
- [ ] Manual testing finds no blockers
### Performance Testing
- [ ] <3s initial load time
- [ ] <250MB memory with 1000 processes
- [ ] >55fps during scrolling
- [ ] <100ms search response
- [ ] No memory leaks in 1-hour session
### Security Testing
- [ ] No raw secrets visible anywhere
- [ ] Logs properly redacted
- [ ] Exports sanitized
- [ ] File access controlled
### Compatibility Testing
- [ ] Chrome/Edge full features work
- [ ] Firefox degraded mode works
- [ ] Safari degraded mode works
- [ ] Mobile responsive design works
### Accessibility Testing
- [ ] Full keyboard navigation
- [ ] Screen reader compatible
- [ ] WCAG 2.1 AA compliant
- [ ] Focus indicators visible
### Documentation
- [ ] All bugs logged with reproduction steps
- [ ] Test report completed
- [ ] Performance metrics documented
- [ ] Sign-off recommendation provided
## Tools and Scripts
Use these tools for testing:
- **Vitest**: Unit and integration tests
- **Playwright**: E2E and browser testing
- **Lighthouse**: Performance and accessibility audit
- **axe DevTools**: Accessibility validation
- **Chrome DevTools**: Memory profiling
- **BrowserStack**: Cross-browser testing
Remember: Your role is to break things before users do. Be thorough, be critical, and don't approve anything that doesn't meet the specifications.
## Hard Rejection Rules
- Fail any task that claims implementation without corresponding code artifacts in `orchestrator-ui/`.
- Fail any task where `npm run build` or relevant tests are not actually executed and reported.
- Fail any task where `npm run dev` only renders placeholder text instead of functional SystemG UI behavior.
- Do not accept documentation-only outputs for implementation stories.
- If unresolved failures remain in the DAG, explicitly mark release readiness as `NO-GO`.