Picking

PickingSystem answers one question: what is under this ray. An entity opts in by carrying a Pickable marker, the application hands the system a batch of rays as PickingQuery objects, and one PickingResult comes back per ray. It is the path from a pointer position to an entity id, and it lives at @woosh/meep-engine/src/engine/graphics3/PickingSystem.js with the rest of the ECS rendering systems.

Three properties change how the calling code is written, and all three are deliberate.

It is asynchronous and batched, on purpose

pick(queries) takes an array and returns a promise. There is no synchronous entry point, and adding one is not on the table. The backend today is a CPU bounding-volume hierarchy that could answer inside the call; the API refuses to, because the backend it is meant to grow into reads entity ids back off the GPU’s visibility buffer - an answer that is inherently a frame late and arrives for a whole batch at once. A synchronous entry point would have to be broken to get there, and every caller broken with it.

Write the calling code as if the answer arrives later, because it will:

• The world can change while the question is out. Check dataset.entityExists(entity) before acting on a result.
• Batch what you can. Asking for four rays in one call is one round trip, not four.

It tests bounding boxes, not triangles

A hit means the ray entered the entity’s world axis-aligned bounding box. That is the precision available for free: Shade already maintains a world bounding box per mesh as part of drawing it, so picking needs no CPU-side copy of the geometry. Going finer would mean keeping triangles for every pickable entity, which is exactly the duplication Pickable exists to keep small.

Two consequences worth designing around:

• A ray that starts inside a box reports a distance of 0 for that entity, and nothing can beat it.
• The result is the nearest box the ray entered - not the nearest surface, and not necessarily the thing you were interested in. A prop with no gameplay meaning standing in front of a chest means “the prop”, not “the chest”. That is the same answer an id read off a pixel would give, which is why the API commits to it.

The index is refreshed on query, not per frame

Nothing pays for picking on a frame where nobody picks. The system holds no per-frame update work at all: the first thing pick() does is walk its leaves, move each one to where its entity now is, and hand a leaf to any entity whose model has arrived since the last look. Because that happens inside the query, there is no window in which the index disagrees with the scene - and therefore no class of bug where a pick lands on where something used to be.

Setting it up

PickingSystem takes one collaborator: the MeshSystem instance, which is where bounds come from.

import { EngineHarness }  from "@woosh/meep-engine/src/engine/EngineHarness.js";
import { MeshSystem }     from "@woosh/meep-engine/src/engine/graphics3/MeshSystem.js";
import { PickingSystem }  from "@woosh/meep-engine/src/engine/graphics3/PickingSystem.js";
import { load_model_scene_bundle } from "@woosh/meep-engine/src/engine/asset/load_model_scene_bundle.js";

const engine = await EngineHarness.bootstrap({
    configuration(config, engine) {
        const meshes = new MeshSystem(
            engine.graphics,
            EngineHarness.shadeScene(engine),
            url => load_model_scene_bundle(engine.assetManager, url),
        );

        config.addSystem(meshes);
        config.addSystem(new PickingSystem(meshes));
    },
});

MeshSystem also needs the glTF and image-bitmap asset loaders registered on the configuration before it can produce anything - meshes & materials covers that side. Two things follow from the picking constructor itself:

• Only entities MeshSystem tracks can be picked. An entity enters MeshSystem by carrying SGMesh + Transform64, and gets bounds once its model has loaded. An entity built directly from a MeshletGeometry and a ShadedGeometry component takes the other mesh path, is not in MeshSystem, and never enters the picking index.
• PickingSystem.dependencies is [Pickable, Transform64]. A Pickable entity with no Transform64 is never linked at all.

PickingSystem3 is a deprecated re-export of the same class. Import PickingSystem.

The Pickable marker

import Entity        from "@woosh/meep-engine/src/engine/ecs/Entity.js";
import { Pickable }  from "@woosh/meep-engine/src/engine/graphics3/Pickable.js";
import { SGMesh }    from "@woosh/meep-engine/src/engine/graphics/ecs/mesh-v2/aggregate/SGMesh.js";
import { Transform64 } from "@woosh/meep-engine/src/engine/ecs/transform/Transform64.js";

const model = new SGMesh();
model.url = "/models/crate.glb";

const transform = new Transform64();
transform.setTranslation(3, 0, -2);

new Entity()
    .add(model)
    .add(transform)
    .add(new Pickable())
    .build(engine.entityManager.dataset);

Pickable carries no data and has nothing to configure - the picking shape comes from what the entity actually looks like, so there is no second way for it to disagree with the model. It is paid for per entity that asks, which is the point: a level is mostly scenery nobody will ever click on, and scenery costs nothing here.

It serializes through PickableSerializationAdapter (already in populateEngineSerializationRegistry). The adapter writes no bytes: what has to survive a save is that the entity opted in, and the presence of the component is that.

Queries

import { PickingQuery } from "@woosh/meep-engine/src/engine/graphics3/PickingQuery.js";

const query = PickingQuery.from(origin, direction, 100);

PickingQuery.from(origin, direction, max_distance) copies both vectors, so a caller can reuse scratch vectors across a whole batch.

The unit-length requirement is not a style preference: result distances are measured along the direction, so a direction of any other length rescales every distance the batch reports, and rescales max_distance with it. GraphicsEngine.viewportProjectionRay writes a unit direction, so a ray built from the viewport is already correct.

Results

import { PickingResult } from "@woosh/meep-engine/src/engine/graphics3/PickingResult.js";

The contract is positional. A batch always answers with one result per query, in the order the queries were given, and a ray that hit nothing still produces a row - so the two arrays line up by index and a caller never has to search for the answer to its own question. distance being Infinity on a miss means an unfiltered batch still sorts and compares sensibly.

Each query yields the single nearest hit. There is no “everything the ray touched” mode, no filter callback, and no layer mask - if you need to exclude something, exclude it by not giving it Pickable, or filter the entity id after the fact.

A click, to an entity

GraphicsEngine supplies both halves of the conversion. normalizeViewportPoint(input, result) turns viewport pixels (origin top-left) into a clip-space point (-1 to 1, +Y up), and viewportProjectionRay(x, y, source, direction) turns that clip-space point into a world-space ray through the one camera the graphics engine owns - the camera CameraSystem copies the active Camera entity onto. Both write into vectors you own, and viewportProjectionRay writes a unit direction, which is what PickingQuery requires.

import Vector2 from "@woosh/meep-engine/src/core/geom/Vector2.js";
import Vector3 from "@woosh/meep-engine/src/core/geom/Vector3.js";
import { PickingQuery }  from "@woosh/meep-engine/src/engine/graphics3/PickingQuery.js";
import { PickingSystem } from "@woosh/meep-engine/src/engine/graphics3/PickingSystem.js";

const picking = engine.entityManager.getSystem(PickingSystem);

const clip      = new Vector2();
const origin    = new Vector3();
const direction = new Vector3();

engine.devices.pointer.on.tap.add(async (position) => {
    engine.graphics.normalizeViewportPoint(position, clip);
    engine.graphics.viewportProjectionRay(clip.x, clip.y, origin, direction);

    const [result] = await picking.pick([
        PickingQuery.from(origin, direction, 100),
    ]);

    if (!result.found) {
        return;
    }

    const dataset = engine.entityManager.dataset;

    if (!dataset.entityExists(result.entity)) {
        // the level went away, or that entity did, while the question was out
        return;
    }

    console.log("picked", result.entity, "at", result.distance, result.position);
});

on.tap sends (position, event), where position is the pointer device’s own live Vector2 - safe to read synchronously, which is all the first two lines do.

To ignore clicks that landed on UI rather than on the game field, compare engine.devices.pointer.getTargetElement() against engine.graphics.domElement before building the ray. TooltipComponentSystem (src/engine/graphics3/TooltipComponentSystem.js) does exactly that and is the reference consumer of this API in the engine: it builds a ray from the pointer, awaits a one-query batch every tick, and acts on whatever comes back without checking whether the pointer moved in the meantime - the answer describes where the pointer was, the next tick asks about where it is now.

Entities that are not in the index yet

An entity can be Pickable and still not be findable, because its model has not loaded. Such an entity holds no leaf in the index and is looked at again on every query, which is what lets it appear without anything having to notice the moment it did. The reverse happens too: if an entity’s model goes away under it, its leaf is dropped and it goes back to waiting rather than losing its opt-in.

picking.indexed_count reports how many entities the index currently stands for. It is 0 until the first pick() - the index is built on query - and stays behind by however many pickable entities are still waiting on their models. Treat it as a diagnostic, not as a count of pickable entities.

What picking does not cover

Picking is one narrow tool: screen ray in, entity id out, bounding-box precision. Three neighbouring questions go somewhere else entirely.

Terrain picking does not come through here. Terrain is not a MeshSystem entity, so it is never in the picking index at all. A terrain hit is an exact raycast against the terrain’s own height representation: Terrain.raycastFirstSync(surfacePoint, ox, oy, oz, dx, dy, dz) fills a SurfacePoint3 with a position and a normal, synchronously. pick(x, y, graphics, terrain, callback) in @woosh/meep-engine/src/engine/ecs/grid/pick.js wraps the whole thing - it does the viewport-to-ray conversion itself and hands the callback the hit in grid coordinates as well as world coordinates. Reach for it whenever the answer you want is a place rather than an entity; it is more precise than picking, not less.

Physics raycasts are a different world. Spatial queries run against colliders in the physics world, and they are synchronous, exact against each shape, accept a filter(entity, collider), and report surface normals. Reach for physics.raycast when the question is about the simulated world - line of sight, a projectile, a ground probe, “what would this bullet hit”. Reach for PickingSystem when the question is about what the player is pointing at on screen and the answer you want is an entity id. An entity can have a collider and no Pickable, or the reverse; the two indexes are unrelated.

Finally, picking is not a general CPU raycast against a mesh. For triangle-accurate intersection against one specific primitive there is ShadedGeometry.query_raycast_nearest(contact, ray, transform_matrix4) - contact is a SurfacePoint3, ray a 6-tuple [ox, oy, oz, dx, dy, dz], and transform_matrix4 a 16-element matrix array. It requires a decoded Geometry: a MeshletGeometry keeps its triangles quantized inside meshlet pages, so a ShadedGeometry placed from a loaded model cannot be asked this without decoding it first via geometry_build_from_meshlet_geometry. It is a tool for tooling, not for gameplay.

See also

• Rendering overview - the GraphicsEngine facade that supplies normalizeViewportPoint and viewportProjectionRay, and the rest of the rendering systems.
• Meshes & materials - MeshSystem, SGMesh, and the loaders picking depends on for bounds.
• Spatial queries - physics.raycast and the collider world, for the questions picking does not answer.