Class BaseForExecOpBindJoinWithRequestOps<QueryType extends Query,MemberType extends FederationMember>

java.lang.Object

se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.BaseForExecOps

se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.UnaryExecutableOpBase

se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.UnaryExecutableOpBaseWithBatching

se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.BaseForExecOpBindJoinWithRequestOps<QueryType,MemberType>

All Implemented Interfaces:

StatsProvider, ExecutableOperator, UnaryExecutableOp

Direct Known Subclasses:

BaseForExecOpBindJoinSPARQL, ExecOpBindJoinBRTPF

public abstract class BaseForExecOpBindJoinWithRequestOps<QueryType extends Query,MemberType extends FederationMember>
extends UnaryExecutableOpBaseWithBatching

A generic implementation of the bind join algorithm that uses executable request operators for performing the requests to the federation member. The implementation is generic in the sense that it works with any type of request operator. Each concrete implementation that extends this base class needs to implement the createExecutableReqOp(Iterable) function to create the request operators with the types of requests that are specific to that concrete implementation. This implementation is capable of separating out each input solution mapping that assigns a blank node to any of the join variables. Then, such solution mappings are not even considered when creating the requests because they cannot have any join partners in the results obtained from the federation member. Of course, in case the algorithm is used with outer-join semantics, these solution mappings are still returned to the output (without joining them with anything). Another capability of this implementation is that, instead of simply using every input batch of solution mappings to directly create a corresponding bind-join request, this implementation can split the input batch into smaller batch for the requests. On top of that, in case a request operator fails, this implementation automatically reduces the batch size for requests and, then, tries to re-process (with the reduced request batch size) the input solution mappings for which the request operator failed. A potential downside of the latter capability is that, if this algorithm has to execute multiple requests per input batch, then these requests are executed sequentially.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	BaseForExecOpBindJoinWithRequestOps.MyIntermediateResultElementSink
protected class	BaseForExecOpBindJoinWithRequestOps.MyIntermediateResultElementSinkOuterJoin

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DEFAULT_BATCH_SIZE
protected final MemberType	fm
protected static final int	minimumRequestBlockSize	The minimum value to which requestBlockSize can be reduced.
protected int	numberOfRequestOpsUsed
protected final QueryType	query
protected int	requestBlockSize	The number of solution mappings that this operator uses for each of the bind join requests.
protected boolean	requestBlockSizeWasReduced
protected ExecutableOperatorStats	statsOfFirstReqOp
protected ExecutableOperatorStats	statsOfLastReqOp
protected final boolean	useOuterJoinSemantics
protected final Set<org.apache.jena.sparql.core.Var>	varsInPatternForFM

Fields inherited from class se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.UnaryExecutableOpBaseWithBatching

batchSize, collectedInputSolMaps

Fields inherited from class se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.BaseForExecOps

collectExceptions

Constructor Summary

Constructors

Constructor	Description
BaseForExecOpBindJoinWithRequestOps(QueryType query, MemberType fm, boolean useOuterJoinSemantics, int batchSize, boolean collectExceptions)
BaseForExecOpBindJoinWithRequestOps(QueryType query, MemberType fm, boolean useOuterJoinSemantics, Set<org.apache.jena.sparql.core.Var> varsInPatternForFM, int batchSize, boolean collectExceptions)

Method Summary

Modifier and Type	Method	Description
protected void	_concludeExecution(List<SolutionMapping> batchOfSolMaps, IntermediateResultElementSink sink, ExecutionContext execCxt)	Implementations of this function need to process the given batch of solution mappings as last input, conclude the execution of this operator, and send the remaining result elements (if any) to the given sink.
protected void	_processBatch(List<SolutionMapping> batchOfSolMaps, IntermediateResultElementSink sink, ExecutionContext execCxt)	Implementations of this function need to process the given batch of solution mappings as input and send the produced result elements (if any) to the given sink.
protected void	_processJoinableInput(Iterable<SolutionMapping> joinableInputSMs, IntermediateResultElementSink sink, ExecutionContext execCxt)
protected void	_processWithoutSplittingInputFirst(List<SolutionMapping> joinableInputSMs, IntermediateResultElementSink sink, ExecutionContext execCxt)
protected abstract NullaryExecutableOp	createExecutableReqOp(Iterable<SolutionMapping> solMaps)	Implementations of this function should create an executable operator that can perform a bind join request in which the query of this bind join operator is combined with the given solution mappings.
protected ExecutableOperatorStatsImpl	createStats()
protected void	extractUnjoinableInputSMs(Iterable<SolutionMapping> solMaps, Iterable<org.apache.jena.sparql.core.Var> potentialJoinVars, List<SolutionMapping> unjoinable, List<SolutionMapping> joinable)	Splits the given collection of solution mappings into two such that the first list contains all the solution mappings that are guaranteed not to have any join partner and the second list contains the rest of the given input solution mappings (which may have join partners).
protected boolean	reduceRequestBlockSize()	Reduces requestBlockSize to its current value divided by 2 if the resulting value would still be greater or equal to minimumRequestBlockSize.
void	resetStats()

Methods inherited from class se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.UnaryExecutableOpBaseWithBatching

_concludeExecution, _process

Methods inherited from class se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.UnaryExecutableOpBase

_process, concludeExecution, getStats, process, process

Methods inherited from class se.liu.ida.hefquin.engine.queryplan.executable.impl.ops.BaseForExecOps

getExceptionsCaughtDuringExecution, recordExceptionCaughtDuringExecution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface se.liu.ida.hefquin.engine.queryplan.executable.ExecutableOperator

getExceptionsCaughtDuringExecution

Field Details

DEFAULT_BATCH_SIZE

public static final int DEFAULT_BATCH_SIZE

See Also:

• Constant Field Values

query

protected final QueryType extends Query query

fm

protected final MemberType extends FederationMember fm

useOuterJoinSemantics

protected final boolean useOuterJoinSemantics

varsInPatternForFM

protected final Set<org.apache.jena.sparql.core.Var> varsInPatternForFM

requestBlockSize

protected int requestBlockSize

The number of solution mappings that this operator uses for each of the bind join requests. This number may be adapted at runtime.

minimumRequestBlockSize

protected static final int minimumRequestBlockSize

The minimum value to which requestBlockSize can be reduced.

See Also:

• Constant Field Values

requestBlockSizeWasReduced

protected boolean requestBlockSizeWasReduced

numberOfRequestOpsUsed

protected int numberOfRequestOpsUsed

statsOfFirstReqOp

protected ExecutableOperatorStats statsOfFirstReqOp

statsOfLastReqOp

protected ExecutableOperatorStats statsOfLastReqOp

Constructor Details

BaseForExecOpBindJoinWithRequestOps

public BaseForExecOpBindJoinWithRequestOps(QueryType query,
 MemberType fm,
 boolean useOuterJoinSemantics,
 Set<org.apache.jena.sparql.core.Var> varsInPatternForFM,
 int batchSize,
 boolean collectExceptions)

Parameters:

varsInPatternForFM - may be used by sub-classes to provide the set of variables that occur in the graph pattern that the bind join evaluates at the federation member; sub-classes that cannot extract this set in their constructor may pass null as value for this argument; if provided, this implementation can filter out input solution mappings that contain blank nodes for the join variables and, thus, cannot be joined with the solution mappings obtained via the requests

batchSize - this value must not be smaller than minimumRequestBlockSize

BaseForExecOpBindJoinWithRequestOps

public BaseForExecOpBindJoinWithRequestOps(QueryType query,
 MemberType fm,
 boolean useOuterJoinSemantics,
 int batchSize,
 boolean collectExceptions)

Parameters:

batchSize - this value must not be smaller than minimumRequestBlockSize

Method Details

_processBatch

protected void _processBatch(List<SolutionMapping> batchOfSolMaps,
 IntermediateResultElementSink sink,
 ExecutionContext execCxt)
                      throws ExecOpExecutionException

Description copied from class: UnaryExecutableOpBaseWithBatching

Implementations of this function need to process the given batch of solution mappings as input and send the produced result elements (if any) to the given sink. If an exception occurs while processing the batch, then this exception needs to be thrown.

Specified by:

_processBatch in class UnaryExecutableOpBaseWithBatching

Throws:

ExecOpExecutionException

_concludeExecution

protected void _concludeExecution(List<SolutionMapping> batchOfSolMaps,
 IntermediateResultElementSink sink,
 ExecutionContext execCxt)
                           throws ExecOpExecutionException

Description copied from class: UnaryExecutableOpBaseWithBatching

Implementations of this function need to process the given batch of solution mappings as last input, conclude the execution of this operator, and send the remaining result elements (if any) to the given sink. Notice that the given batch of solution mappings may be empty! If an exception occurs during this process, then this exception needs to be thrown.

Specified by:

_concludeExecution in class UnaryExecutableOpBaseWithBatching

Throws:

ExecOpExecutionException

_processJoinableInput

protected void _processJoinableInput(Iterable<SolutionMapping> joinableInputSMs,
 IntermediateResultElementSink sink,
 ExecutionContext execCxt)
                              throws ExecOpExecutionException

Throws:

ExecOpExecutionException

_processWithoutSplittingInputFirst

protected void _processWithoutSplittingInputFirst(List<SolutionMapping> joinableInputSMs,
 IntermediateResultElementSink sink,
 ExecutionContext execCxt)
                                           throws ExecOpExecutionException

Throws:

ExecOpExecutionException

reduceRequestBlockSize

protected boolean reduceRequestBlockSize()

Reduces requestBlockSize to its current value divided by 2 if the resulting value would still be greater or equal to minimumRequestBlockSize. In this case, this function returns true. Otherwise, the function returns false without reducing requestBlockSize.

createExecutableReqOp

protected abstract NullaryExecutableOp createExecutableReqOp(Iterable<SolutionMapping> solMaps)

Implementations of this function should create an executable operator that can perform a bind join request in which the query of this bind join operator is combined with the given solution mappings. The operator created by this function should throws exceptions instead of collecting them.

extractUnjoinableInputSMs

protected void extractUnjoinableInputSMs(Iterable<SolutionMapping> solMaps,
 Iterable<org.apache.jena.sparql.core.Var> potentialJoinVars,
 List<SolutionMapping> unjoinable,
 List<SolutionMapping> joinable)

Splits the given collection of solution mappings into two such that the first list contains all the solution mappings that are guaranteed not to have any join partner and the second list contains the rest of the given input solution mappings (which may have join partners). Typically, the solution mappings that are guaranteed not to have join partners are the ones that have a blank node for one of the join variables.

resetStats

public void resetStats()

Specified by:

resetStats in interface StatsProvider

Overrides:

resetStats in class UnaryExecutableOpBaseWithBatching

createStats

protected ExecutableOperatorStatsImpl createStats()

Overrides:

createStats in class UnaryExecutableOpBaseWithBatching