Quick Data Conversion
quick-conversion.RdFast, flexible and precise conversion of common data objects, without method dispatch and extensive checks:
qDF,qDTandqTBLconvert vectors, matrices, higher-dimensional arrays and suitable lists to data frame, data.table and tibble, respectively.qMconverts vectors, higher-dimensional arrays, data frames and suitable lists to matrix.mctlandmrtlcolumn- or row-wise convert a matrix to list, data frame or data.table. They are used internally byqDF/qDT/qTBL,dapply,BY, etc...qFconverts atomic vectors to factor (documented on a separate page).as_numeric_factor,as_integer_factor, andas_character_factorconvert factors, or all factor columns in a data frame / list, to character or numeric (by converting the levels).
Usage
# Converting between matrices, data frames / tables / tibbles
qDF(X, row.names.col = FALSE, keep.attr = FALSE, class = "data.frame")
qDT(X, row.names.col = FALSE, keep.attr = FALSE, class = c("data.table", "data.frame"))
qTBL(X, row.names.col = FALSE, keep.attr = FALSE, class = c("tbl_df","tbl","data.frame"))
qM(X, row.names.col = NULL , keep.attr = FALSE, class = NULL, sep = ".")
# Programmer functions: matrix rows or columns to list / DF / DT - fully in C++
mctl(X, names = FALSE, return = "list")
mrtl(X, names = FALSE, return = "list")
# Converting factors or factor columns
as_numeric_factor(X, keep.attr = TRUE)
as_integer_factor(X, keep.attr = TRUE)
as_character_factor(X, keep.attr = TRUE)Arguments
- X
a vector, factor, matrix, higher-dimensional array, data frame or list.
mctlandmrtlonly accept matrices,as_numeric_factor,as_integer_factorandas_character_factoronly accept factors, data frames or lists.- row.names.col
can be used to add an column saving names or row.names when converting objects to data frame using
qDF/qDT/qTBL.TRUEwill add a column"row.names", or you can supply a name e.g.row.names.col = "variable". IfXis a named atomic vector, a length 2 vector of names can be supplied, e.g.,qDF(fmean(mtcars), c("car", "mean")). WithqM, the argument has the opposite meaning, and can be used to select one or more columns in a data frame/list which will be used to create the rownames of the matrix e.g.qM(iris, row.names.col = "Species"). In this case the column(s) can be specified using names, indices, a logical vector or a selector function. See Examples.- keep.attr
logical.
FALSE(default) yields a hard / thorough object conversion: All unnecessary attributes are removed from the object yielding a plain matrix / data.frame / data.table.FALSEyields a soft / minimal object conversion: Only the attributes 'names', 'row.names', 'dim', 'dimnames' and 'levels' are modified in the conversion. Other attributes are preserved. See alsoclass.- class
if a vector of classes is passed here, the converted object will be assigned these classes. If
NULLis passed, the default classes are assigned:qMassigns no class,qDFa class"data.frame", andqDTa classc("data.table", "data.frame"). Ifkeep.attr = TRUEandclass = NULLand the object already inherits the default classes, further inherited classes are preserved. See Details and the Example.- sep
character. Separator used for interacting multiple variables selected through
row.names.col.- names
logical. Should the list be named using row/column names from the matrix?
- return
an integer or string specifying what to return. The options are:
Int. String Description 1 "list" returns a plain list 2 "data.frame" returns a plain data.frame 3 "data.table" returns a plain data.table
Details
Object conversions using these functions are maximally efficient and involve 3 consecutive steps: (1) Converting the storage mode / dimensions / data of the object, (2) converting / modifying the attributes and (3) modifying the class of the object:
(1) is determined by the choice of function and the optional row.names.col argument. Higher-dimensional arrays are converted by expanding the second dimension (adding columns, same as as.matrix, as.data.frame, as.data.table).
(2) is determined by the keep.attr argument: keep.attr = TRUE seeks to preserve the attributes of the object. Its effect is like copying attributes(converted) <- attributes(original), and then modifying the "dim", "dimnames", "names", "row.names" and "levels" attributes as necessitated by the conversion task. keep.attr = FALSE only converts / assigns / removes these attributes and drops all others.
(3) is determined by the class argument: Setting class = "myclass" will yield a converted object of class "myclass", with any other / prior classes being removed by this replacement. Setting class = NULL does NOT mean that a class NULL is assigned (which would remove the class attribute), but rather that the default classes are assigned: qM assigns no class, qDF a class "data.frame", and qDT a class c("data.table", "data.frame"). At this point there is an interaction with keep.attr: If keep.attr = TRUE and class = NULL and the object converted already inherits the respective default classes, then any other inherited classes will also be preserved (with qM(x, keep.attr = TRUE, class = NULL) any class will be preserved if is.matrix(x) evaluates to TRUE.)
The default keep.attr = FALSE ensures hard conversions so that all unnecessary attributes are dropped. Furthermore in qDF/qDT/qTBL the default classes were explicitly assigned. This is to ensure that the default methods apply, even if the user chooses to preserve further attributes. For qM a more lenient default setup was chosen to enable the full preservation of time series matrices with keep.attr = TRUE. If the user wants to keep attributes attached to a matrix but make sure that all default methods work properly, either one of qM(x, keep.attr = TRUE, class = "matrix") or unclass(qM(x, keep.attr = TRUE)) should be employed.
Value
qDF - returns a data.frameqDT - returns a data.tableqTBL - returns a tibbleqM - returns a matrixmctl, mrtl - return a list, data frame or data.table qF - returns a factoras_numeric_factor - returns X with factors converted to numeric (double) variablesas_integer_factor - returns X with factors converted to integer variablesas_character_factor - returns X with factors converted to character variables
Examples
## Basic Examples
mtcarsM <- qM(mtcars) # Matrix from data.frame
mtcarsDT <- qDT(mtcarsM) # data.table from matrix columns
mtcarsTBL <- qTBL(mtcarsM) # tibble from matrix columns
head(mrtl(mtcarsM, TRUE, "data.frame")) # data.frame from matrix rows, etc..
#> Mazda RX4 Mazda RX4 Wag Datsun 710 Hornet 4 Drive Hornet Sportabout
#> mpg 21.00 21.000 22.80 21.400 18.70
#> cyl 6.00 6.000 4.00 6.000 8.00
#> disp 160.00 160.000 108.00 258.000 360.00
#> hp 110.00 110.000 93.00 110.000 175.00
#> drat 3.90 3.900 3.85 3.080 3.15
#> wt 2.62 2.875 2.32 3.215 3.44
#> Valiant Duster 360 Merc 240D Merc 230 Merc 280 Merc 280C Merc 450SE
#> mpg 18.10 14.30 24.40 22.80 19.20 17.80 16.40
#> cyl 6.00 8.00 4.00 4.00 6.00 6.00 8.00
#> disp 225.00 360.00 146.70 140.80 167.60 167.60 275.80
#> hp 105.00 245.00 62.00 95.00 123.00 123.00 180.00
#> drat 2.76 3.21 3.69 3.92 3.92 3.92 3.07
#> wt 3.46 3.57 3.19 3.15 3.44 3.44 4.07
#> Merc 450SL Merc 450SLC Cadillac Fleetwood Lincoln Continental
#> mpg 17.30 15.20 10.40 10.400
#> cyl 8.00 8.00 8.00 8.000
#> disp 275.80 275.80 472.00 460.000
#> hp 180.00 180.00 205.00 215.000
#> drat 3.07 3.07 2.93 3.000
#> wt 3.73 3.78 5.25 5.424
#> Chrysler Imperial Fiat 128 Honda Civic Toyota Corolla Toyota Corona
#> mpg 14.700 32.40 30.400 33.900 21.500
#> cyl 8.000 4.00 4.000 4.000 4.000
#> disp 440.000 78.70 75.700 71.100 120.100
#> hp 230.000 66.00 52.000 65.000 97.000
#> drat 3.230 4.08 4.930 4.220 3.700
#> wt 5.345 2.20 1.615 1.835 2.465
#> Dodge Challenger AMC Javelin Camaro Z28 Pontiac Firebird Fiat X1-9
#> mpg 15.50 15.200 13.30 19.200 27.300
#> cyl 8.00 8.000 8.00 8.000 4.000
#> disp 318.00 304.000 350.00 400.000 79.000
#> hp 150.00 150.000 245.00 175.000 66.000
#> drat 2.76 3.150 3.73 3.080 4.080
#> wt 3.52 3.435 3.84 3.845 1.935
#> Porsche 914-2 Lotus Europa Ford Pantera L Ferrari Dino Maserati Bora
#> mpg 26.00 30.400 15.80 19.70 15.00
#> cyl 4.00 4.000 8.00 6.00 8.00
#> disp 120.30 95.100 351.00 145.00 301.00
#> hp 91.00 113.000 264.00 175.00 335.00
#> drat 4.43 3.770 4.22 3.62 3.54
#> wt 2.14 1.513 3.17 2.77 3.57
#> Volvo 142E
#> mpg 21.40
#> cyl 4.00
#> disp 121.00
#> hp 109.00
#> drat 4.11
#> wt 2.78
head(qDF(mtcarsM, "cars")) # Adding a row.names column when converting from matrix
#> cars mpg cyl disp hp drat wt qsec vs am gear carb
#> 1 Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4
#> 2 Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4
#> 3 Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1
#> 4 Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1
#> 5 Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0 3 2
#> 6 Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0 3 1
head(qDT(mtcars, "cars")) # Saving row.names when converting data frame to data.table
#> cars mpg cyl disp hp drat wt qsec vs am
#> <char> <num> <num> <num> <num> <num> <num> <num> <num> <num>
#> 1: Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1
#> 2: Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1
#> 3: Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1
#> 4: Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0
#> 5: Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0
#> 6: Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0
#> gear carb
#> <num> <num>
#> 1: 4 4
#> 2: 4 4
#> 3: 4 1
#> 4: 3 1
#> 5: 3 2
#> 6: 3 1
head(qM(iris, "Species")) # Examples converting data to matrix, saving information
#> Sepal.Length Sepal.Width Petal.Length Petal.Width
#> setosa 5.1 3.5 1.4 0.2
#> setosa 4.9 3.0 1.4 0.2
#> setosa 4.7 3.2 1.3 0.2
#> setosa 4.6 3.1 1.5 0.2
#> setosa 5.0 3.6 1.4 0.2
#> setosa 5.4 3.9 1.7 0.4
head(qM(GGDC10S, is.character)) # as rownames
#> Year AGR MIN MAN PU
#> BWA.SSA.Sub-saharan Africa.VA 1960 NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA 1961 NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA 1962 NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA 1963 NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA 1964 16.30154 3.494075 0.7365696 0.1043936
#> BWA.SSA.Sub-saharan Africa.VA 1965 15.72700 2.495768 1.0181992 0.1350976
#> CON WRT TRA FIRE GOV
#> BWA.SSA.Sub-saharan Africa.VA NA NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA NA NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA NA NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA NA NA NA NA NA
#> BWA.SSA.Sub-saharan Africa.VA 0.6600454 6.243732 1.658928 1.119194 4.822485
#> BWA.SSA.Sub-saharan Africa.VA 1.3462312 7.064825 1.939007 1.246789 5.695848
#> OTH SUM
#> BWA.SSA.Sub-saharan Africa.VA NA NA
#> BWA.SSA.Sub-saharan Africa.VA NA NA
#> BWA.SSA.Sub-saharan Africa.VA NA NA
#> BWA.SSA.Sub-saharan Africa.VA NA NA
#> BWA.SSA.Sub-saharan Africa.VA 2.341328 37.48229
#> BWA.SSA.Sub-saharan Africa.VA 2.678338 39.34710
head(qM(gv(GGDC10S, -(2:3)), 1:3, sep = "-")) # plm-style rownames
#> AGR MIN MAN PU CON WRT TRA
#> BWA-VA-1960 NA NA NA NA NA NA NA
#> BWA-VA-1961 NA NA NA NA NA NA NA
#> BWA-VA-1962 NA NA NA NA NA NA NA
#> BWA-VA-1963 NA NA NA NA NA NA NA
#> BWA-VA-1964 16.30154 3.494075 0.7365696 0.1043936 0.6600454 6.243732 1.658928
#> BWA-VA-1965 15.72700 2.495768 1.0181992 0.1350976 1.3462312 7.064825 1.939007
#> FIRE GOV OTH SUM
#> BWA-VA-1960 NA NA NA NA
#> BWA-VA-1961 NA NA NA NA
#> BWA-VA-1962 NA NA NA NA
#> BWA-VA-1963 NA NA NA NA
#> BWA-VA-1964 1.119194 4.822485 2.341328 37.48229
#> BWA-VA-1965 1.246789 5.695848 2.678338 39.34710
qDF(fmean(mtcars), c("cars", "mean")) # Data frame from named vector, with names
#> cars mean
#> 1 mpg 20.090625
#> 2 cyl 6.187500
#> 3 disp 230.721875
#> 4 hp 146.687500
#> 5 drat 3.596563
#> 6 wt 3.217250
#> 7 qsec 17.848750
#> 8 vs 0.437500
#> 9 am 0.406250
#> 10 gear 3.687500
#> 11 carb 2.812500
# mrtl() and mctl() are very useful for iteration over matrices
# Think of a coordninates matrix e.g. from sf::st_coordinates()
coord <- matrix(rnorm(10), ncol = 2, dimnames = list(NULL, c("X", "Y")))
# Then we can
for (d in mrtl(coord)) {
cat("lon =", d[1], ", lat =", d[2], fill = TRUE)
# do something complicated ...
}
#> lon = -0.256752 , lat = 0.1006336
#> lon = 0.08769642 , lat = 0.1453606
#> lon = 0.4257396 , lat = -1.924576
#> lon = 0.3697442 , lat = -0.4193368
#> lon = 0.3010184 , lat = -0.8559307
rm(coord)
## Factors
cylF <- qF(mtcars$cyl) # Factor from atomic vector
cylF
#> [1] 6 6 4 6 8 6 8 4 4 6 6 8 8 8 8 8 8 4 4 4 4 8 8 8 8 4 4 4 8 6 8 4
#> Levels: 4 6 8
# Factor to numeric conversions
identical(mtcars, as_numeric_factor(dapply(mtcars, qF)))
#> [1] TRUE