R S4 overloading of subscript operator for base matrix class

Question

I have defined a S4 class called FilterCommandIndices this way:

setClass('FilterCommandIndices',
     representation(rows='logical', cols='logical')
     )

It just contains two slots, logical vectors indicating whether a given row or column is included in a set. Then I want to overload the subscript operator for it to work with the previous class. My initial implementation goes like this:

setMethod('[', c(x='ANY', i='FilterCommandIndices', j='missing'),
      function(x, i, j, ..., drop=TRUE) {
        if (nrow(x) != length(getRows(i)) || ncol(x) != length(getCols(i))) {
          stop('Incorrect number of dimensions')
        } else {
          return(x[getRows(i), getCols(i)])
        }
      })

In the previous code, getRows() and getCols() are the corresponding trivial accessors for the FilterCommandIndices class:

setGeneric('getRows', function(object) standardGeneric('getRows'))

setMethod('getRows', 'FilterCommandIndices',
      function(object) {
        return(object@rows)
      })

setGeneric('getCols', function(object) standardGeneric('getCols'))

setMethod('getCols', 'FilterCommandIndices',
      function(object) {
        return(object@cols)
      })

I thought that by using the ANY signature, I could make this simple class and operator combination to work even in base classes, like a simple matrix. Although it seems to work on an example class...

d> bar1
An object of class "FilterCommandIndices"
Slot "rows":
[1] FALSE  TRUE  TRUE  TRUE  TRUE

Slot "cols":
[1] TRUE TRUE TRUE TRUE TRUE

d> mockMethylSet
MethylSet (storageMode: lockedEnvironment)
assayData: 5 features, 5 samples 
  element names: Meth, Unmeth 
phenoData: none
Annotation
Preprocessing
  Method: unknown
  minfi version: unknown
  Manifest version: unknown

d> mockMethylSet[bar1]
MethylSet (storageMode: lockedEnvironment)
assayData: 4 features, 5 samples 
  element names: Meth, Unmeth 
phenoData: none
Annotation
Preprocessing
   Method: unknown
  minfi version: unknown
  Manifest version: unknown

...but not in the base matrix class:

d> zeroDetectionP
   S1 S2 S3 S4 S5
F1  0  0  0  0  0
F2  0  0  0  0  0
F3  0  0  0  0  0
F4  0  0  0  0  0
F5  0  0  0  0  0

d> zeroDetectionP[bar1]
Error en zeroDetectionP[bar1] : invalid subscript type 'S4'

In the setMethod docs, it says that you can define methods using base classes in the signature, what made me think that I could implement something like the previous method using an ANY signature and failing only when the subscripted object does not implement nrow(), ncol() or the operator '['.

Any help or hint will be much appreciated.

UPDATE: Set a specific signature method (as @hadley suggested)

If I set a method with a specific signature for the matrix base class...

d> getMethod('[', c(x='matrix', i='FilterCommandIndices', j='missing'))
Method Definition:

function (x, i, j, ..., drop = TRUE) 
{
    return(x) # Dummy method
}

... and then try again to subscript a matrix...

d> zeroDetectionP[bar1]
Error en zeroDetectionP[bar1] : invalid subscript type 'S4'

... the error is still there.

UPDATE: Minimal reproducible example

I have tried to write a minimal scenario where I could reproduce the previous scenario. This is, I think, a simple scenario that gives the same error. As @hadley suggested, it might be due to the subscript being a primitive.

library(methods)

setClass('Indices', representation(rows='logical', cols='logical'))

setMethod('[', c(x='ANY', i='Indices', j='missing'),
      function(x, i, j, ..., drop=FALSE) {
        return(x[i@rows, i@cols])
      })

setClass('M', representation(m='matrix'))

setMethod('[', c(x='M', i='logical', j='logical'),
      function(x, i, j, ..., drop=FALSE) {
        return(x@m[i, j])
      })

anIndicesObject <- new('Indices', rows=c(TRUE, TRUE, FALSE), cols=c(FALSE, TRUE, FALSE))
aMatrix <- matrix(1:9, nrow=3)
aMobject <- new('M', m=aMatrix)

aMobject # Prints the M object
aMobject[c(TRUE, TRUE, FALSE), c(TRUE, TRUE, TRUE)] # Susbcript by logical vector
aMobject[anIndicesObject] # Subscript by Indices class
aMatrix[anIndicesObject] # Subscript a matrix by Indices --> ERROR

Answer 1

Even more minimal reproducible example:

library(methods)

setClass("Indices", representation(rows = "logical", cols = "logical"))
x <- new("Indices", rows=c(TRUE, TRUE, FALSE), cols=c(FALSE, TRUE, FALSE))
m <- matrix(1:9, nrow=3)
m[x]

# Method doesn't appear to be found for signature x = "ANY"
show_s4 <- function(x, i, j, ..., drop=FALSE) "S4"
setMethod("[", signature(x = "ANY", i = "Indices", j = "missing"), show_s4)
m[x]

# Or even with explicit "matrix" in signature
setMethod("[", signature(x = "matrix", i = "Indices"), show_s4)
m[x]
# Error in m[x] : invalid subscript type "S4"

# Describing all arguments also fails
getGeneric("[")
setMethod(
  "[", 
  signature(x = "matrix", i = "Indices", j = "missing", drop = "missing"),
  show_s4
)
m[x]

I think the key is this sentence in ?Methods:

An S4 method alone will not be seen if the S3 generic function is called directly. However, primitive functions and operators are exceptions: The internal C code will look for S4 methods if and only if the object is an S4 object.

Since matrix is not an S4 class, you can not dispatch on it in internally generic methods (like [)