3D Mouse Picking for Bevy
A Bevy plugin for 3D mouse picking, making it easy to interact with 3D geometry using your mouse or any other ray casting source! The plugin includes built-in mouse events, highlighting, selection state, multi-window support, and a debug cursor, all of which are opt-in.
Expect breaking changes in master
branch - contributions are welcome!
Features
- Multiple picking sources
- Mouse (relative to supplied camera)
- Screen space coordinates (relative to supplied camera)
- Manually defined ray using a transform
- Pick Data
- Pick depth
- Pick coordinates (in world space)
- Surface normal at the pick ray intersection
- Vertex coordinates of the intersected triangle
- Mesh Interaction
- Mouseover and click events
- Configurable color highlighting for hovering and selection
- Mesh selection state management
- 3D Debug cursor to show pick intersection and surface normal
- Picking Groups (associate a picking source with a set of meshes)
- Multi window support
Demo
To run the 3d_scene
example - a modified version of the Bevy
example of the same name - clone this repository and run:
cargo run --example 3d_scene --features="example_deps"
Note that by default this plugin only depends on bevy's render
feature to minimize dependency count and build time, and allow for wasm support. This is why the feature flag is needed to run examples, which need the winit and wgpu features to run.
Getting Started
It only takes a few lines to get mouse picking working in your Bevy application using this plugin; check out the 3d_scene example for a minimal implementation and demo! The following sections will walk you through what is needed to get the plugin working, and how everything fits together.
Setup
Add the plugin to your dependencies in Cargo.toml
= "0.3.5"
Import the plugin:
use *;
Add it to your App::build() in the plugins section of your Bevy app:
.add_plugin
Marking Entities for Picking
For simple use cases, you will probably be using the mouse to pick items in a 3d scene. You will can mark your camera with a default PickSource component:
.with
Now all you have to do is mark any mesh entities with the PickableMesh
component:
.with
Interacting with Meshes
To get mouseover and mouseclick events, as well as built-in highlighting and selection state, you will need to add the InteractablePickingPlugin
plugin. This is intentionally left optional, in case you only need pick intersection results.
// Add this below the PickingPlugin line
.add_plugin
See the Pick Interactions section for more details on the features this provides.
You will need to add the InteractableMesh
component to entities to use these features.
.with
.with
If you want a mesh to highlight when you hover, add the HighlightablePickMesh
component:
// InteractableMesh component is a prerequisite for this to work
.with
If you also want to select meshes and keep them highlighted when clicked with the left mouse button, add the SelectablePickMesh
component:
// InteractableMesh component is a prerequisite for this to work
.with
Pick Groups
Pick groups allow you to associate meshes with a ray casting source, and produce a pick result for each group. For simple use cases, such as a single 3d view and camera, you can ignore this.
For those simple cases, you can just use Group::default()
any time a Group
is required. This will assign the PickableMesh
or PickSource
to picking group 0.
Pick groups are useful in cases such as multiple windows, where you want each window to have its own picking source (cursor relative to that window's camera), and each window might have a different set of meshes that this picking source can intersect. The primary window might assign the camera and all relavent meshes to pick group 0, while the secondary window uses pick group 1 for these. See the multiple_windows example for implementation details.
Constraints
- Only one PickSource can be assigned to a
Group
- A PickableMesh can be assigned to one or more
Group
s - The result of running the picking system is an ordered list of all intersections of each
PickSource
with thePickableMesh
s in thatGroup
. The ordered list of intersections are stored byGroup
,HashMap<Group, Vec<PickIntersection>>
Getting Pick Data
Mesh picking intersection are reported in world coordinates. You can use the PickState
resource to either get the topmost entity, or a list of all entities sorted by distance (near -> far) under the cursor:
Alternatively, and perhaps more idiomatic to the Bevy ECS system, you can get the intersections for entities that have the PickableMesh
component using:
pickable_entity.intersection;
Pick Interactions
Run the events
example to see mouseover and mouseclick events in action:
cargo run --example events
The InteractableMesh
component stores mouseover event state, mouseclick event state (left, right, and middle buttons), and hover state.
Selection State
If you're using the SelectablePickMesh
component for selection, you can access the selection state by querying your selectable entities and accessing the .selected()
function.
Plugin Parameters
If you're using the built in HighlightablePickMash
component for highlighting, you can change the colors by accessing the PickHighlightParams
and setting the colors:
// Example Bevy system to set the highlight colors
Debug
You can enable a debug cursor that will place a sphere at the intersection, with a tail pointing normal to the surface. Just add the DebugPickingPlugin
to the App::build()
in your Bevy program:
.add_plugin
Bounding Sphere Optimization
This plugin has the ability to accelerate picking with bounding spheres; this can make picking as much as 30 times faster! This speeds up the picking process by first checking to see if the picking source intersects a mesh's bounding sphere before going through every triangle in the mesh. To enable bounding spheres, you can use the builder pattern to pass a handle to your mesh into the .with_bounding_sphere()
function:
.with;
This will run a system in Bevy to automatically compute the bounding sphere of the supplied mesh.You can see an example of bounding spheres used in the stress_test
example. Please be aware that the API for this feature is likely to change over coming releases.
License
This project is licensed under the MIT license.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in bevy_mod_picking by you, shall be licensed as MIT, without any additional terms or conditions.