r - apply a function on two lists with different number of elements

Question

I have these two list generated from a data.table object and I would like to apply a function over them.

The lists look similar to this:

>list1

$AA
            C
1: 0.07519183

$BB
           C
1: 0.7174377

$CC
           C
1: 0.1620897

$DD
          C
1: 0.184746

>list2

$AA
           P1        P2        P3       P4        P5      P6
1: 0.04770305 0.1624142 0.2899578 0.029753 0.1070376 0.17549

$BB
          P1        P2        P3        P4        P5        P6
1: 0.7174377 0.5965736 0.2561482 0.2561482 0.2561482 0.1997866

$CC
          P1       P2       P3         P4        P5        P6
1: 0.0317663 0.139877 0.139877 0.05305057 0.1620897 0.2189595

$DD
         P1        P2        P3        P4        P5        P6
1: 0.184746 0.4246214 0.2704228 0.1070376 0.3215871 0.1519672

The function that I would like to apply is the following:

fun <- function(x,y){(sum(x>=y)+1)/(length(y)+1)}

I've tried:

new.list <- mapply(fun, list1, list2)

but it gives an error: >= only defined for equally-sized data frames. I could repeat the values in list1 to avoid this error but is there another way?

Answer 1

using mapply will be inherently slow since this will require iterating over each item and performing each operation over and over.

A much faster option would be do flatten the lists and use vectorized operations.

Flatten the lists.

# Convert list1 to a vector
L1 <- as.vector(unlist(list1))

# Convert list2 to a matrix
L2 <- as.matrix(rbindlist(list2))

results <- (rowSums(L1 >= L2) + 1) / (ncol(L2)+1)

## Add names if needed
names(results) <- names(list2)

results

Compare

fun <- function(x, y) (sum( x[[1]] >= y) + 1) / (length(y) + 1)
results.m <- mapply(fun, list1, list2)

identical(results, results.m)

Much faster than mapply

library(microbenchmark)
microbenchmark(MatrixStyle=(rowSums(L1 >= L2) + 1) / (ncol(L2)+1),  
               MapplyStyle=mapply(fun, list1, list2))

Unit: microseconds
        expr     min       lq   median       uq      max neval
 MatrixStyle   9.560  11.4125  13.9925  15.2890   34.205   100
 MapplyStyle 639.037 674.2595 697.1065 723.8985 5938.127   100