These snippets use a Pandas DataFrame named log; follow the event-data import examples to create it. After filtering, compare process statistics or run process discovery.
PM4Py offers a variety of specific methods to filter event logs.
Start with this six-case log. Each filter below operates on its input log; filtering a subset does not replace the original.
import pm4py
log = pm4py.read_xes("tests/input_data/running-example.xes")
print("Input:", log["case:concept:name"].nunique(), "cases,", len(log), "events")The following section presents various methods for filtering event logs based on time frames. Each example operates directly on a Pandas DataFrame. You might be interested in retaining only the traces that fall within a specific time interval, such as from January 1 to January 15, 2011.
import pm4py
import pandas
if __name__ == "__main__":
filtered_log: pandas.DataFrame = pm4py.filter_time_range(log, "2011-01-01 00:00:00", "2011-01-15 23:59:59", mode='traces_contained'){'first case IDs': ['4'],
'input cases': 6,
'input events': 42,
'retained cases': 1,
'retained events': 5}Additionally, it is possible to keep the traces that intersect with a time interval.
import pm4py
import pandas
if __name__ == "__main__":
filtered_log: pandas.DataFrame = pm4py.filter_time_range(log, "2011-01-01 00:00:00", "2011-01-15 23:59:59", mode='traces_intersecting'){'first case IDs': ['3', '2', '1', '6', '5', '4'],
'input cases': 6,
'input events': 42,
'retained cases': 6,
'retained events': 42}So far, only trace-based filtering techniques have been discussed. However, there is also a method to keep the events that fall within a specific timeframe.
import pm4py
import pandas
if __name__ == "__main__":
filtered_log: pandas.DataFrame = pm4py.filter_time_range(log, "2011-01-01 00:00:00", "2011-01-15 23:59:59", mode='events'){'first case IDs': ['3', '2', '1', '6', '5', '4'],
'input cases': 6,
'input events': 42,
'retained cases': 6,
'retained events': 26}This filter allows you to keep only traces with durations that fall within a specified interval. In the examples, traces between 1 and 10 days are kept. Note that the time parameters are given in seconds.
import pm4py
import pandas
if __name__ == "__main__":
filtered_log: pandas.DataFrame = pm4py.filter_case_performance(log, 86400, 864000){'first case IDs': ['2', '1', '6', '4'],
'input cases': 6,
'input events': 42,
'retained cases': 4,
'retained events': 20}In general, PM4Py can filter a log or a DataFrame based on start activities. First, you may need to identify the starting activities, for which code snippets are provided. The snippet discovers start activities and filters the DataFrame.
log_start is a dictionary where the key is the activity and the value is the number of occurrences.
import pm4py
import pandas
if __name__ == "__main__":
log_start: dict[str, int] = pm4py.get_start_activities(log)
filtered_log: pandas.DataFrame = pm4py.filter_start_activities(log, ["register request"]){'first case IDs': ['3', '2', '1', '6', '5', '4'],
'input cases': 6,
'input events': 42,
'retained cases': 6,
'retained events': 42}PM4Py also allows filtering by end activities. This filter keeps only traces that end with a specified set of activities. First, you may need to identify the end activities, for which a code snippet is provided.
import pm4py
import pandas
if __name__ == "__main__":
end_activities: dict[str, int] = pm4py.get_end_activities(log)
filtered_log: pandas.DataFrame = pm4py.filter_end_activities(log, ["pay compensation"]){'first case IDs': ['3', '2', '6'],
'input cases': 6,
'input events': 42,
'retained cases': 3,
'retained events': 19}A variant refers to a set of cases that share the same control-flow perspective, meaning a set of cases that follow the same sequence of activities in the same order. Variants are represented as tuples of activity names; each tuple describes an ordered sequence. To retrieve the variants from the log, you can use the following code snippet:
import pm4py
if __name__ == "__main__":
variants: dict = pm4py.get_variants(log){('register request', 'check ticket', 'examine casually', 'decide', 'pay compensation'): 1,
('register request', 'examine thoroughly', 'check ticket', 'decide', 'reject request'): 1}To filter by a specific collection of variants, use the following code snippet:
import pm4py
import pandas
if __name__ == "__main__":
filtered_log: pandas.DataFrame = pm4py.filter_variants(log, [("register request", "examine casually", "check ticket", "decide", "pay compensation")]){'first case IDs': ['6'],
'input cases': 6,
'input events': 42,
'retained cases': 1,
'retained events': 5}Other variant-based filters are available. For example, filters on the top-k variants retain only the cases following one of the k most frequent variants:
import pm4py
import pandas
if __name__ == "__main__":
log: pandas.DataFrame = pm4py.read_xes("tests/input_data/receipt.xes")
k: int = 2
filtered_log: pandas.DataFrame = pm4py.filter_variants_top_k(log, k){'first case IDs': ['case-10024',
'case-10025',
'case-10059',
'case-10061',
'case-10065',
'case-10066',
'case-10067',
'case-10068'],
'input cases': 1434,
'input events': 8577,
'retained cases': 836,
'retained events': 5016} The variant coverage filter retains only those traces that follow the top variants in the log, under the condition that each variant covers a specified percentage of cases. For instance, if min_coverage_percentage=0.4 and we have a log with 1000 cases, where 500 are variant 1, 400 are variant 2, and 100 are variant 3, the filter will keep only traces for variants 1 and 2.
import pm4py
import pandas
if __name__ == "__main__":
log: pandas.DataFrame = pm4py.read_xes("tests/input_data/receipt.xes")
perc: float = 0.1
filtered_log: pandas.DataFrame = pm4py.filter_variants_by_coverage_percentage(log, perc){'first case IDs': ['case-10024',
'case-10025',
'case-10059',
'case-10065',
'case-10066',
'case-10067',
'case-10068',
'case-10069'],
'input cases': 1434,
'input events': 8577,
'retained cases': 713,
'retained events': 4278}Filtering by attribute values allows you to:
Examples of attributes include the resource (typically contained in the org:resource attribute) and the activity (usually found in the concept:name attribute). To get the list of resources and activities contained in the log, you can use the following code.
import pm4py
if __name__ == "__main__":
log = pm4py.read_xes("tests/input_data/receipt.xes")
activities: dict[str, int] = pm4py.get_event_attribute_values(log, "concept:name")
resources: dict[str, int] = pm4py.get_event_attribute_values(log, "org:resource"){'activities (excerpt)': {'Confirmation of receipt': 1434,
'T02 Check confirmation of receipt': 1368,
'T04 Determine confirmation of receipt': 1307,
'T06 Determine necessity of stop advice': 1416},
'resources (excerpt)': {'Resource01': 1228,
'Resource02': 580,
'Resource03': 552,
'Resource04': 483}}To filter traces containing or not containing a given list of resources, use the following code:
import pm4py
import pandas
if __name__ == "__main__":
log = pm4py.read_xes("tests/input_data/receipt.xes")
tracefilter_log_pos: pandas.DataFrame = pm4py.filter_event_attribute_values(log, "org:resource", ["Resource10"], level="case", retain=True)
tracefilter_log_neg: pandas.DataFrame = pm4py.filter_event_attribute_values(log, "org:resource", ["Resource10"], level="case", retain=False){'excluded events': 6905, 'retained events': 1672}You can also keep only the events performed by a given list of resources, trimming the cases accordingly. Use the following code for this:
import pm4py
import pandas
if __name__ == "__main__":
log = pm4py.read_xes("tests/input_data/receipt.xes")
tracefilter_log_pos: pandas.DataFrame = pm4py.filter_event_attribute_values(log, "org:resource", ["Resource10"], level="event", retain=True)
tracefilter_log_neg: pandas.DataFrame = pm4py.filter_event_attribute_values(log, "org:resource", ["Resource10"], level="event", retain=False){'excluded events': 8248, 'retained events': 329}Filtering by numeric attribute values offers options similar to filtering by string attributes. First, we import the log, then filter to keep only events satisfying a numeric range between 34 and 36. An additional filter can be applied to retain only cases with at least one event within the specified range. If you're interested in cases with a specific activity, such as "Add penalty," having an amount between 34 and 500, the following code snippet can be used:
import os
import pandas as pd
import pm4py
import pandas
if __name__ == "__main__":
df: pandas.DataFrame = pd.read_csv(os.path.join("tests", "input_data", "roadtraffic100traces.csv"))
df = pm4py.format_dataframe(df)
from pm4py.algo.filtering.pandas.attributes import attributes_filter
filtered_df_events: pandas.DataFrame = attributes_filter.apply_numeric_events(df, 34, 36,
parameters={attributes_filter.Parameters.CASE_ID_KEY: "case:concept:name", attributes_filter.Parameters.ATTRIBUTE_KEY: "amount"})
filtered_df_cases: pandas.DataFrame = attributes_filter.apply_numeric(df, 34, 36,
parameters={attributes_filter.Parameters.CASE_ID_KEY: "case:concept:name", attributes_filter.Parameters.ATTRIBUTE_KEY: "amount"})
filtered_df_cases = attributes_filter.apply_numeric(df, 34, 500,
parameters={attributes_filter.Parameters.CASE_ID_KEY: "case:concept:name", attributes_filter.Parameters.ATTRIBUTE_KEY: "amount",
attributes_filter.Parameters.STREAM_FILTER_KEY1: "concept:name",
attributes_filter.Parameters.STREAM_FILTER_VALUE1: "Add penalty"}){'case filter rows': 299, 'event filter rows': 27, 'input events': 390}The between filter transforms the event log by identifying subcases that span from a source activity to a target activity. This is useful for analyzing behavior between two activities, such as throughput time, activity inclusion, or conformance levels. The filter between two activities is applied as follows:
import pm4py
import pandas
if __name__ == "__main__":
log: pandas.DataFrame = pm4py.read_xes("tests/input_data/running-example.xes")
filtered_log: pandas.DataFrame = pm4py.filter_between(log, "check ticket", "decide"){'first case IDs': ['3##@@0', '3##@@1', '2##@@0', '1##@@0', '6##@@0', '5##@@0', '5##@@1', '5##@@2'],
'input cases': 6,
'input events': 42,
'retained cases': 9,
'retained events': 21}The case size filter retains only the cases in the log that have a number of events within a user-specified range. This filter can be used to eliminate cases that are too short (possibly incomplete or outliers) or too long (indicating excessive rework). The case size filter can be applied as follows:
import pm4py
import pandas
if __name__ == "__main__":
log: pandas.DataFrame = pm4py.read_xes("tests/input_data/running-example.xes")
filtered_log: pandas.DataFrame = pm4py.filter_case_size(log, 5, 10){'first case IDs': ['3', '2', '1', '6', '4'],
'input cases': 6,
'input events': 42,
'retained cases': 5,
'retained events': 29}The rework filter identifies cases where a given activity has been repeated. For instance, it can be used to search for cases that contain at least two occurrences of the activity "reinitiate request."
import pm4py
import pandas
if __name__ == "__main__":
log: pandas.DataFrame = pm4py.read_xes("tests/input_data/running-example.xes")
filtered_log: pandas.DataFrame = pm4py.filter_activities_rework(log, "reinitiate request", 2){'first case IDs': ['5'],
'input cases': 6,
'input events': 42,
'retained cases': 1,
'retained events': 13}The path performance filter identifies cases in which the duration of a given path between two activities falls within a specified range. This is useful for identifying cases where a significant amount of time has passed between two activities. The filter is applied as follows:
import pm4py
import pandas
if __name__ == "__main__":
log: pandas.DataFrame = pm4py.read_xes("tests/input_data/running-example.xes")
filtered_log: pandas.DataFrame = pm4py.filter_paths_performance(log, ("decide", "pay compensation"), 2*86400, 10*86400){'first case IDs': ['2', '3', '6'],
'input cases': 6,
'input events': 42,
'retained cases': 3,
'retained events': 19}