Generalize USMT Dash and add a containerized demo mode
Remove all environment-specific data and make the dashboard adoptable anywhere, then add a demo that runs the real PowerShell services. - Strip cached device data, internal knowledge-base articles and screenshots, server/site names and the old project notes; replace the knowledge base with generic sample articles and diagrams - Move every environment value into Config/usmt-dash.psd1 - Route all infrastructure access in the worker and Get-Software.ps1 through a provider (Providers/SCCM.ps1), with a config-driven asset inventory view instead of a hard-coded custom class - Add Providers/Demo.ps1: a deterministic simulated fleet whose migrations progress in real time, and Demo/New-DemoData.ps1 to seed it - Server: -Demo makes the API read-only; add get-config and get-logs; serve static files only from web/, Documentation/ and an allowlist of Data/ files with a path-traversal guard; gzip large responses - Mirror both services' console output to Data/Logs via a Write-Host proxy, and add a terminal panel to the dashboard that streams it - Demo sandbox (web/demo/demo.js): visitor changes to tags, articles and images are kept in the visitor's browser and echoed in the terminals; nothing a visitor does reaches the server - Vendor marked, highlight.js and html2pdf.js instead of loading CDNs - Dockerfile and compose file for hosting the demo (Coolify-ready)
This commit is contained in:
Johan Sandoval committed
commit
1c531bc2ab
56 files changed
+10232
No files matched your search
@@ -0,0 +1,201 @@
|
||||
# USMT Dash
|
||||
|
||||
A real-time migration dashboard for Windows device-refresh projects. It tracks every old-to-new
|
||||
PC migration driven by ConfigMgr (SCCM) and USMT: who's online, how much disk is free, where
|
||||
each backup and restore is, and which specialty apps still need to go onto the new machine.
|
||||
|
||||
It's written in **PowerShell** end to end: an `HttpListener` web server with a JSON API, a
|
||||
polling worker that queries SCCM's SQL database, Active Directory and the devices themselves,
|
||||
and a single-file HTML/JS front end. It needs no web framework, database server or runtime
|
||||
beyond PowerShell.
|
||||
|
||||
> **Live demo:** the hosted demo runs the real server and worker in a container against a
|
||||
> simulated 200-device fleet. Open the **Terminals** panel at the bottom of the page to watch
|
||||
> them work.
|
||||
|
||||
|
||||
|
||||
## Why it exists
|
||||
|
||||
On a large hardware refresh, a technician checking one device in the ConfigMgr console (the
|
||||
association, whether the backup started, whether the box is even on the network) took about a
|
||||
minute. USMT Dash brings that down to under a second for every device at once, and gives a
|
||||
team a shared view of the whole refresh.
|
||||
|
||||
Built and run during a real enterprise Windows 11 refresh, where the team's daily build rate
|
||||
went from about 10 to 25 devices a day and the project closed out about 1,900 machines against
|
||||
a 1,300-device scope.
|
||||
|
||||
## Features
|
||||
|
||||
- **Tracked Devices**: technicians register devices under tags (a name, a date batch, a
|
||||
department). The worker only sweeps tagged devices. Tags can be flagged *Skip USMT* for
|
||||
inventory-only tracking, and `SOURCE, DEST` lines create manual pairs for hardware exceptions.
|
||||
- **USMT Status**: every migration grouped by state (*Not Started, Backup In Progress, Backup
|
||||
Completed, Restore In Progress, Completed, Not Associated*), with ping, free space and the
|
||||
latest scanstate/loadstate log line. Each column can be sorted, filtered and copied.
|
||||
- **Smart status engine**: combines the ConfigMgr `MigrationStatus` with whether the USMT
|
||||
progress log in `SMSTSLog` is actively being written. Completed pairs are cached and skip
|
||||
network checks on later cycles. A reverted association resets the cache.
|
||||
- **Network checks that tell the truth**: resolves *all* A records, pings each one, and
|
||||
validates reverse DNS. This separates *Online*, *Timed Out*, *Not Found* and *DNS Issue*
|
||||
(a stale record pointing at a different machine).
|
||||
- **Missing software**: compares installed software on source and destination, filters out the
|
||||
baseline image using a pattern list, groups multi-part products into suites, and lists the
|
||||
specialty apps not yet on the new device.
|
||||
- **Device details**: AD OU and groups, hardware, disk, RAM, client health, location,
|
||||
department, users. Shown for both sides of a migration.
|
||||
- **Custom View**: build your own table from any field, with drag-to-reorder, resizable and
|
||||
filterable columns. Saved per browser.
|
||||
- **Workbench**: personal "stacks" of devices for the cart in front of you.
|
||||
- **Knowledge base**: Markdown articles with an editor, image manager, PDF export, badges and
|
||||
click-to-copy links. Renaming an image rewrites every article that links to it. Articles
|
||||
named after an app are linked from that app in the software view.
|
||||
- **Business-hours scheduling**: the worker cycles every few minutes during working hours,
|
||||
sleeps overnight, and makes a midnight "pitstop" for the daily software refresh.
|
||||
*Force Update* and *Force Full Refresh* skip the wait.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
|  |  |
|
||||
|  |  |
|
||||
|
||||
## Architecture
|
||||
|
||||
|
||||
|
||||
The server and worker never talk to each other directly. `Data\` is the entire interface:
|
||||
the worker writes JSON state files, the server serves them, and the dashboard's "force
|
||||
refresh" buttons drop flag files that the worker watches for.
|
||||
|
||||
| Component | What it does |
|
||||
|---|---|
|
||||
| `USMT-Server.ps1` | `System.Net.HttpListener` server: static files, a JSON API (17 routes), IP allowlist, gzip |
|
||||
| `USMT-Worker.ps1` | Polling loop: builds the working set from the tags, sweeps each device, works out its status, writes `Data\*.json` |
|
||||
| `Get-Software.ps1` | Per-device installed-software lookup with a 24-hour cache (used by the worker and the API) |
|
||||
| `Providers\SCCM.ps1` | Real data source: ConfigMgr module, site SQL database, ADSI, DNS/ICMP, admin shares |
|
||||
| `Providers\Demo.ps1` | Simulated data source used by the demo (see below) |
|
||||
| `Config\usmt-dash.psd1` | Every environment-specific setting |
|
||||
| `web\index.html` | The dashboard (single-file HTML/JS, about 4,500 lines) |
|
||||
| `web\demo\demo.js` | Terminal panel, plus the demo-mode browser sandbox |
|
||||
|
||||
### API
|
||||
|
||||
| Method & path | Purpose |
|
||||
|---|---|
|
||||
| `GET /` and static files | Dashboard, assets, `Data\` state files, `Documentation\` |
|
||||
| `GET /get-config` | Mode flags for the front end |
|
||||
| `GET /get-logs?source=server\|worker&after=<id>` | Tail of the service console output (terminal panel) |
|
||||
| `POST /force-update` · `POST /force-full` | Wake the worker now (full = also refresh software/inventory caches) |
|
||||
| `GET /get-blacklist` | Baseline-app pattern list |
|
||||
| `GET /get-all-tags` · `GET /get-tracked?tech=` · `GET /get-tag-meta?tech=` | Read tracking tags |
|
||||
| `POST /update-tracked` | Set a tag's devices and *Skip USMT* flag |
|
||||
| `GET /get-software?name=` | Installed software for one device |
|
||||
| `GET /get-docs-list` · `POST /save-doc` · `POST /delete-doc` | Knowledge base (delete archives to `Documentation\Archive`) |
|
||||
| `GET /list-images` · `POST /upload-image` · `POST /rename-image` · `POST /delete-image` | Knowledge-base images |
|
||||
|
||||
## Demo mode
|
||||
|
||||
The public demo has to show the whole dashboard working without real infrastructure, and
|
||||
without letting strangers change what other visitors see.
|
||||
|
||||
- **It's the real code.** The container runs the unmodified `USMT-Server.ps1` and
|
||||
`USMT-Worker.ps1` under PowerShell 7 with `-Demo`. The worker loads `Providers\Demo.ps1`
|
||||
instead of `Providers\SCCM.ps1`, and that's the only difference.
|
||||
- **A believable fleet.** The demo provider derives each device's hardware, users, location,
|
||||
department and software from a hash of its name. Each migration moves through every stage
|
||||
on a 24-hour clock, so the dashboard keeps changing. Devices go offline, have stale DNS,
|
||||
run low on disk, and fail backups at realistic rates.
|
||||
- **Read-only API.** In demo mode the server refuses every route that writes (and
|
||||
`get-software`, which takes free-text input) with `403`. The terminal output is the real
|
||||
console of both processes, mirrored to `Data\Logs` by a `Write-Host` proxy.
|
||||
- **Visitor changes stay in their browser.** `web\demo\demo.js` intercepts the dashboard's
|
||||
write requests and keeps the visitor's tags, articles and images in IndexedDB, merged over
|
||||
the server's data. Their actions print in the terminal panel exactly as the services would,
|
||||
marked with a small dot. Nobody else sees them, and nothing a visitor types reaches the
|
||||
server, so there's nothing to moderate or inject. **DEMO > Reset my changes** clears it.
|
||||
|
||||
Without `-Demo`, none of this applies: `demo.js` passes every request straight through and
|
||||
the dashboard works against the real backend.
|
||||
|
||||
## Running it
|
||||
|
||||
### Demo (Docker / Coolify)
|
||||
|
||||
```bash
|
||||
docker compose up -d --build # http://localhost:8080
|
||||
```
|
||||
|
||||
On **Coolify**, create a resource from this repository using the *Dockerfile* (or *Docker
|
||||
Compose*) build pack, set the exposed port to **8080**, and attach a domain. The container
|
||||
reseeds the simulated fleet on every start and needs no volumes or secrets. Set `TZ` to show
|
||||
times in your timezone.
|
||||
|
||||
### Demo (local PowerShell 7)
|
||||
|
||||
```powershell
|
||||
pwsh ./Demo/New-DemoData.ps1 -Force
|
||||
pwsh ./USMT-Worker.ps1 -Demo # terminal 1
|
||||
pwsh ./USMT-Server.ps1 -Demo # terminal 2 -> http://localhost:8080
|
||||
```
|
||||
|
||||
### Live, against your environment
|
||||
|
||||
Requirements: Windows PowerShell 5.1, the ConfigMgr console (for the `ConfigurationManager`
|
||||
module), read access to the site database, AD read access, and admin-share access to devices.
|
||||
|
||||
1. Edit `Config\usmt-dash.psd1`: site code, SMS Provider, SQL server and database, and (if
|
||||
you extend hardware inventory with an asset/location class) the view and column map
|
||||
under `AssetInventory`.
|
||||
2. Start the worker, then the server:
|
||||
```powershell
|
||||
.\USMT-Worker.ps1 # add -v for per-device tracing
|
||||
.\USMT-Server.ps1 # binding http://*:8080/ may need: netsh http add urlacl url=http://*:8080/ user=DOMAIN\account
|
||||
```
|
||||
3. Optionally restrict access by listing allowed IPs or hostnames in `Data\Whitelisted-IPs.txt`.
|
||||
4. Put baseline-image apps in `Data\Blacklisted-Apps.txt` so only specialty software shows as
|
||||
missing. See `Demo\Blacklisted-Apps.txt` for the syntax (plain patterns, `Suite = a*, b*`,
|
||||
`Rule: DEVICE* = app*`, `Combo: Label = a* && b*`).
|
||||
|
||||
### Adapting it to a different toolset
|
||||
|
||||
All infrastructure access goes through a provider. To support another platform (Intune, a
|
||||
different CMDB, PDQ, ...), add `Providers\<Name>.ps1`, implement these functions, and set
|
||||
`Provider = '<Name>'` in the config:
|
||||
|
||||
| Function | Returns |
|
||||
|---|---|
|
||||
| `Initialize-Provider` | One-time setup (import modules, ...) |
|
||||
| `Open-ProviderCycle` / `Close-ProviderCycle` | Per-cycle session (connections) |
|
||||
| `pingDevice <name>` | `Online`, `Timed Out`, `Not Found` or `DNS Issue` |
|
||||
| `Resolve-DeviceIP <name>` | Current IP, or `$null` |
|
||||
| `Get-DiskSpace <name>` | `@{ Total = '238 GB'; Free = '80 GB' }` |
|
||||
| `Get-LogInfo <name> <logName>` | Latest USMT progress log line (via the shared parser) |
|
||||
| `Get-DeviceAssociations <name>` | Objects with `SourceComputer`, `DestinationComputer`, `MigrationStatus` (0/1/2) |
|
||||
| `Get-DeviceInventory <name> <session>` | Hashtable of detail fields (OU, Groups, Model, RAM, Location, ...) |
|
||||
| `Get-InstalledSoftware <name>` | Rows of `Name`, `Version`, `Publisher`, `Source`, `InstallDate` |
|
||||
|
||||
`Providers\Demo.ps1` is a complete working example that needs no infrastructure.
|
||||
|
||||
## Project layout
|
||||
|
||||
```
|
||||
USMT-Server.ps1 web server + JSON API
|
||||
USMT-Worker.ps1 polling worker
|
||||
Get-Software.ps1 per-device software lookup + cache
|
||||
Config/usmt-dash.psd1 configuration
|
||||
Lib/Common.ps1 config loader, console-log mirror, shared I/O
|
||||
Providers/SCCM.ps1 ConfigMgr / SQL / AD / SMB data source
|
||||
Providers/Demo.ps1 simulated fleet
|
||||
Demo/ demo seed script + sample app blacklist
|
||||
Documentation/ knowledge-base articles and images
|
||||
web/ dashboard, terminal panel / demo sandbox, vendored libraries
|
||||
Dockerfile, docker/ container for the hosted demo
|
||||
Data/ runtime state (created on first run, not committed)
|
||||
```
|
||||
|
||||
## Third-party code
|
||||
|
||||
Vendored in `web/vendor/` under their own licenses: [marked](https://github.com/markedjs/marked)
|
||||
(MIT), [highlight.js](https://github.com/highlightjs/highlight.js) (BSD-3-Clause),
|
||||
[html2pdf.js](https://github.com/eKoopmans/html2pdf.js) (MIT).
|
||||
Reference in new issue
Block a user