Causal artificial intelligence (AI) · Model guide
Partially observable decision processes
Partially observable Markov decision process (POMDP)
Compare action policies when the system’s true state cannot be observed directly.
Choose actions while maintaining a belief about a hidden state.
Also known as: MDP.
The model in pictures
Partially observable Markov decision process (POMDP)
A model for choosing a sequence of actions when the true situation is only partly known. It tracks beliefs about that situation and weighs possible actions and their outcomes.
How should a care plan change when a noisy observation is concerning?
Circles are possible patient states, rectangles are actions, and hexagons are observations. The selected treat action and concerning observation illustrate the setup; this screenshot does not show a solved policy or a treatment recommendation.
Synthetic example, not real patient, customer or operational data.
Variables and symbols
- State and b
- Stable, worsening or critical; b is the belief, or probability assigned to each state.
- A and O
- A is an action: monitor, treat or escalate. O is an observation: reassuring, concerning or alarming.
- T and R
- T describes state changes after an action. R is the reward score assigned to outcomes and actions.
- Policy (pi)
- The rule for selecting actions. Horizon 2 means planning over two decision steps.
Markov decision process (MDP)
A model for sequential decisions when each new state is observed exactly. This example is the fully observed special case of a partially observable Markov decision process (POMDP).
Which water-management action best balances future outcomes over four decision steps?
The top row lists possible reservoir states; the middle row lists actions; the bottom row records observations that identify the new state exactly. The pictured release selection is a setup choice, not a returned optimal policy.
Synthetic example, not real patient, customer or operational data.
Variables and symbols
- State and b
- Reservoir level is low, normal or high; b shows the initial probability assigned to each level.
- A and O
- A denotes conserve, release or spill actions. O is the exactly observed level after an action.
- T and R
- T describes level changes after an action. R is the model's reward score.
- Policy (pi) and horizon
- A policy selects actions; horizon 4 means four decision steps.
When to use it
Compare care actions when the patient state is only partially observed.
Applications
- Predictive maintenance: choose inspect, service, or wait actions from incomplete condition signals.
- Robotics: select actions while sensors reveal only part of the environment state.
Why it matters
The result ranks policies under declared transitions, observations, and rewards; it does not claim that a decision action is a causal intervention.
Questions you can ask
- Which action has the best expected value at the current belief?
Causal boundary
This family does not expose causal intervention or counterfactual queries. Dependencies, rules, dynamics, and decision actions alone do not establish those semantics.
Profile and resource limits
Finite horizon and stationary state-action rewards. Exact policies use alpha vectors; point-based bounds are not a complete policy. Fully observed MDPs are a profile.
Online uses protected execution with plan and request limits. Check the returned method, exactness, and resource diagnostics for each query.
Existing workspace example
Adaptive Care under Partial Observation
Choose whether to monitor, treat, or escalate when the patient may be stable, worsening, or critical and observations are noisy. Update beliefs and compare exact with point-based policies.
After signing in, open the workspace example gallery and choose this example. Workspace access and actions follow your plan.
Sign in to explore examplesUse it in your application
Released through the native engine and host software development kits (SDKs). Online and SDK resource profiles differ; check the documentation bundled with your exact SDK version.
Read SDK guidance and examples