aqStats.trajCluster that did not transform coordinates before distance matrix was calculated. Thanks to Dan Jaffe.corPlot to carry through “use” option in cor.added a new importUKAQ() function, which supersedes importAURN(), importAQE(), importWAQN(), importSAQN(), importNI() and importLocal(). importUKAQ() brings a lot of new functionality to accessing UK air quality data through {openair}.
importUKAQ() has all of the same arguments as the functions it supersedes, as well as “source” to define the specific network of interest. The “source” argument can either be of length 1 or equal to the length of the “site” argument. This means that importUKAQ() can import statistics from multiple different networks at once.
importUKAQ() allows ratified = TRUE and to_narrow = TRUE simultaneously. This will return a tibble with two columns per observation - “value” containing the concentration and “qc” containing TRUE/FALSE which indicates whether the concentration is validated.
The AQE, WAQN, SAQN and NI networks now allow for data_type = "daqi". At time of writing, only data from 2022 is present.
the site and pollutant arguments of importUKAQ() are respected when data_type is “daqi”, “annual” or “monthly”. The default behaviour to return all available statistics has not changed. (#346)
the various arguments which augment import function outputs now behave more consistently with one another. For example, ratified now respects pollutant and only returns the "_qc" columns for the pollutants of interest.
importAURN(), importAQE(), importWAQN(), importSAQN(), importNI() and importLocal() are still exported by {openair}. These are all simply wrappers around importUKAQ() with forced “source” arguments, and remain for back-compatibility and convenience.
While importKCL() also imports uk air quality data, it is not currently made available through importUKAQ(). Users should continue to import KCL data via importKCL() for the time being.
importMeta() has gained two new “source” options to assist with the new importUKAQ() function:
source = "ukaq" will return metadata for all of the “UKAQ” networks.
source = "all" will return all available metadata (including KCL and Europe).
new function runRegression() for extracting ‘dilution lines’ from air quality and other data. Online manual will be updated with principles and examples.
calendarPlot() now automatically creates its own labels if breaks are specified. For example, c(0, 10, 20) will create the labels c("0 - 10", "10 - 20"). labels can still be used to override the default values. (#341)
Added the w.abbr.len argument to calendarPlot() which controls the length of the weekday abbreviation. This was requested to help people using written Chinese, but will be more broadly useful (e.g., to use “Mon”, “Tue”, “Wed” in place of “M”, “T”, “W”). (#101)
return tibble from timeAverage().
add option panel.gap to leave space between panels in timeAverage day-hour plots. Set to 0 for previous behaviour.
Move regression formula off main plot for polarPlot() for clarity and label slope as ‘m’.
Tweak seasonal trend decomposition using STL to allow the seasonal amplitude to vary more. Affects smoothTrend() and TheilSen().
Added colours recommended by the UK Government Analysis Function (https://analysisfunction.civilservice.gov.uk/policy-store/data-visualisation-colours-in-charts/) to openColours().
polarCluster now prints the cluster contributions and returns a data frame of them in the output.
The order of columns in importUKAQ() will remain consistent (metadata, date, pollutants, meteo) regardless of whether hc is TRUE or FALSE.
quickText() will now automatically capitalise “no” to “NO”. (#343)
The year argument of importMeta() is now respected when source = "kcl" and "europe".
Several of the directional analysis plot family (e.g., polarFreq()) have been refactored to use is.null() or is.na() over missing(). While predominantly an internal change, this should be make these functions easier to use inside of other functions (e.g., function(data, breaks = NA) polarFreq(data, breaks = breaks)) will now run successfully).
For calendarPlot when annotated with ws or wd arrows, use max ws/wd that corresponds to hour of maximum pollutant concentration and not simple the max ws/wd for a day.
summaryPlot() no longer forces time zones to be GMT. (#356)
add option meteo to importAURN() family of functions. By default modelled wind speed, direction and ambient temperature are returned if available, but not if meteo = FALSE.
added a new column to modStats(), “P”, which represents the P-value of the correlation as reported by cor.test().
update processing of ADMS meteorological (.MOP) files to return stability and tidier data.
refined the output of pollutionRose() where the ws2 and wd2 options are provided; instead of the misleading N/E/S/W, the markers become 0, +90, +/-180, -90.
the timeAverage() progress bar is now powered by {cli}/{purrr} and can be silenced using the new progress argument.
the polarCluster() progress bar is now also powered by {cli}/{purrr}.
colours corresponding to the UK daily air quality index (https://uk-air.defra.gov.uk/air-pollution/daqi) have been added as options to openColours().
the {openair} object’s data property returned by the trajLevel() function is now consistently formatted regardless of statistic choice, and contains relevant statistic-specific information (e.g., sigma for "SQTBA").
fixed issue with modStats() such that “method” can now be changed (e.g., to “spearman”).
fixed issue with aqStats() where output wasn’t being properly split by pollutant/type.
fixed issue with conditionalQuantile() where the plot would fail to be produced when “type” was not specified.
fixed issue with cutData() where “season” wouldn’t respect system locale (e.g., would still show “(DJF)” on Italian systems, instead of the correct “(gla)”). Note that the season name itself (e.g., “Winter”) cannot be automatically converted.
fixed the theilSen() silent argument. The message “taking bootstrap samples. please wait” is also now sent via message() rather than print() and only appears once per function call.
fixed issue where trajCluster() proportions would overlap when clustering forward trajectories. Proportions should now appear at the end of cluster paths, regardless of whether the trajectory is back or forward.
fixed issue where not all {openair} plotting functions would properly return an openair S3 object, and that not all data objects were tibbles.
Fixed issue with timeAverage() where date formatting caused problems, possibly due to latest version of R (4.3.0).
do not use native pipe yet - does not work with old versions of R
fix issue with polarDiff() where the resulting openair object did not contain the plot element.
add year as an option to importMeta. This allows the user to select sites that were only open at some point in the chosen year or duration of years.
make sure full daily gravimetric data are returned for PM10 and PM2.5 if available when using importAURN family of functions. These data will be returned as gr_pm2.5 and gr_pm10 if data_type = "daily"
add alpha argument to all polar directional analysis functions. This is mainly for use in openairmaps but may be of general interest for specific use cases.
fix small bug in smoothTrend in returned fit data when data is missing.
importLocal to access locally-managed automatic monitoring datapollutionRose when single number of breaks given (was ignored)import, import.2, importAURNcsv and kernelExceedsigma to options for SQTBA trajectory analysis for control over plume spread assumptions; set to 1.5 km (in one hour)timeAverage for multi-time period averagestimeProp and simplify codeimportAURN as data_type.importAURN family as data_type.importMetacalendarPlot issue when there are gaps between selected months e.g.month = c(1, 2, 12)calendarPlottimeAverage if DSTresolution in polarPlot — now uses predictions at a coarser resolution that are interpolated. Will also speed-up plottingtrajlevel), SQTBA — Simplified Quantitative Transport Bias AnalysispolarPlot when two pollutants supplied for different statisticspolarPlot. This regression approach determines a best fit line when there is error in the ‘x’ and the ‘y’. The openair book will be updated to cover this in more detail.importAURN.importAURN.windRose and pollutionRose.TaylorDiagram for annotation of observed data.timevariation when difference = TRUEtrajCluster issue to do with dplyrimportAURN, importSAQN, importWAQN, importAQE and importNI. New option data_type, which can be “hourly” (default), “annual”, “monthly”, “daily” and “15min” (for SO2). These new data sources should make it much easier to work with long-term time series with many sites. See the openair manual for more details.selectRunning.windRoseselectRunning. Now returns full data frame with a new condition column.month not passed in calendarPlotwindRose where whole period is calmpolarCluster to speed up clustering through option pamonce = 3. This should not appreciably affect results.aqStats due to lubridate time zone issue.TaylorDiagram when group was present.plot.type to summaryPlot to change line style; most useful for vertical lines in time series with plot.type = "h"polarClusterpolarDiff to consider the difference surface between two polar plots.polarCluster to consider clustering of differences in polar plot surfaces.windRose and pass on option for number of significant figures used to annotate plots (dig.lab)percentileRose when statistic = "cpf" and multiple pollutantstimeAverage when type = "season" and avg.time = "season"week to openair default typestimeVariation when considering difference plots with missing datareturnair_temp) to meteorological variables returned from importAURN, importSAQN and importWAQN (using WRF model).importAQE due to strange dplyr join issuesstatistic = "Spearman" to polarPlot as an option when considering two pollutants.method option to corPlot to allow different correlation methods (“pearson”, “spearman” or “kendall”)importAURN, importSAQN, importWAQN, importAQE.importNI to import data from Northern Ireland.ratified. These functions include importAURN, importSAQN, importWAQN, importAQE and importNIimportMetapercentileRose.importWAQN that would intermittently fail; sometimes crashing R.date.format to TheilSen.calendarPlot slowness on MacOSpolarPlotstatistic = "nwr" in polarPlot that implements the Non-parametric Wind Regression based on Henry et al. (2009). The openair implementation is not identical but should yield similar results.importAQE function to import data from Air Quality England sites.importEurope to provide access to some the data from the saqgetr package.TheilSen when no missing data and deseason = TRUEtimeAverage when interval padding dates and date is “Date” class and not “POSIXct”trajCluster, should be n.cluster not n.clusterscutDatatimeAverage, used for avg.time = “season”.to_narrow to importAURN, importSAQN, importKCL and importWAQN to stack data into a tidy format. The data are now returned as a ‘tibble’importSAQN and importWAQN.TheilSen when conditioning and < 6 annual measurementspolarPlot axes.deseason = TRUE in smoothTrend and TheilSen. This replaces simple linear interpolation.smoothTrend when ci = FALSE (no smooth was fitted).importWAQN to access data from the Welsh air quality network.importMeta.importMeta.aqStats use default data.thresh = 0 rather than 75% to ensure summaries are calculatedtimeVariation when statistic - “median”. Revision will result in narrower range.corPlot to plot lower and upper triangles; add lower as an option.timeAverage when interval more than one time unit e.g. “10 day”polarPlot(mydata, cols = "plasma")align to be used in aqStats to determine how rolling means are calculated. Can take the values “centre” (default), “left” and “right”.importAURNpolarAnnulusdplyr.lubridate in timeAverage to improve speed / simplicityopenair objectspolarPlot when statistic = "cpf" when using tibblespolarCluster with exported data (date was not correctly merged to produce single date column)mydata to a ‘tibble’ for easier printing.npoints = NA in trajPlot to suppress plotting of interval points.windRose when ws/wd have different namestimeAverage for wind direction (wd) when statistic = "data.cap"windRose(mydata, col = "cbPalette", breaks = 6). Thanks to Jerry Martin.plot to TheilSen. FALSE can be useful when analysing data to extract the trend components and plot in other ways and when the TheilSen plot is not required.silent to TheilSen to avoid printing updates to trend fitting. By default it is FALSE.timeVariation when more than one pollutantselectByDate.timePlotcalendarPlot to span any time period. The function can now straddle parts of two years or several years.col.arrow to calendarPlot to control colour of the wind speed / direction annotation arrows.selectByDate due to changes in lubridateimportSAQN when no dataimportAURN and point users to importMetascatterPlot when method = "level" due to NSEtidyr in place of reshape2plyr and reshape2percentileRose for method = "cpf" with multiple pollutantspolarCluster resolution to “normal” rather than “fine” to speed up.trajLeveltimeAverage when statistic = "sum" and all data in period was missing (would return 0 rather than NA)conditionalEval plots by using data where there are no missing data for all variables used. This is more important of var.obs and var.mod are supplied and hence additional variables are considered relative to only obs and mod.calendarPlot to do with time zoneswindRose/pollutionRose when two conditioning variables were given (problem in bias correction)angle option to percentileRose to allow wind direction averages for sectors >10 degrees.aqStats if only a few lines of datastatistic to equal “mean”, “median” or “frequency” in scatterPlot, when method = "level"scatterPlot when method = "level" to use tensor interaction to allow for better smoothing when x and y are on different scales.polarPlot and suggest setting force.postive = FALSEmaps Suggests rather than DependswindRoseTheilSen and when no trend information is givenbinData to easily summarise mean and 95% confidence intervals for intervals of a variablebootMeanDF, used to calculate the bootstrap uncertainty in the mean of a vector.importAURNpolarPlot to work with pairwise statistics to compare two pollutants. The function can consider Pearson correlation and slopes from ordinary linear regression, robust regression (using MASS function rlm) and quantile regression (requires the quantreg package to be installed). See open access version of the paper.polarPlot plot resolution to “fine”.windRose problem with some data due to missing datamapdata package to SuggestsimportAURN and importKCL.polarPlot.y.relation is used when there is no grouping in timePlot. To retain the behaviour of earlier versions use relation = "free"type when used with timePropwindRosetimePlotalpha to polarPlot to control transparency of plotted surface. Mostly useful for overlaying polar plots on leaflet maps (see openairmaps package)grid.line option in windRose so that users can control grid spacing, line type and line colourdownload.file can use libcurl for access to https (used in importAURN)avg.time option to summaryPlot to control the averaging times of the time series lines and print.datacap to control whether the data capture % is shown for each interval.selectByDate where a day number would not workwindRose, polarPlot to avoid ambiguous interpretation of wind directionkey.position option to timePlot to control the location of the key.timeVariation when data for some types is missing.trendLevel issue due to dplyrlubridate package in timeAveragesummaryPlot related to dplyr use (would not plot missing data correctly)windRose mean and statistics returned in data (thanks to Dr Ulrich Quass)importKCL when incomplete time series (would drop site code and site name)pollutionRose due to issue with calmssmoothTrend where model uncertainties were not returnedsummaryPlot where missing data would not be shown correctly when date was not ordered in sequencewindRose when comparing two data setsGoogleMapsPlot is deprecated and will be replaced with a better function.corPlotimportAURNimportMeta for AURN - change in database source formatpollutionRose plot warningmodStatstimeAverage where estimating the time interval in input data could be unreliable due to low data availabilitytimeAveragetype “yearseason” (or “seasonyear”). This will split data by every year / season combination, making sure the seasons are contiguous. For example, in winter in the northern hemisphere December 2010 will be considered part of winter 2011, rather than winter 2010. Thanks to Ralf Weisse for the suggestion.importAURN and importMeta when users are within an organisational network. Problems likely due to move from http to https and SSL Certificates.TheilSen bug when two types.dplyrconditionalEval to avoid plot error.importAURNimportAURN meta data in help function.scatterPlot to have control over plot symbol fill and colour (for symbols 21 to 25). Use cols and fill to control.scatterPlot can now fit more than one linear equation when there is a grouping variabletype such as month, year etc.timeProp, remove box.width optionwindRose bias correction (thanks to Eric Christensen)statistic = "r" in polarPlot for comparing polar plot correlation surfaces between two pollutants using Gaussian kernel weighting.type = "daylight" when time zone not UTC.cutDaylight as a separate function (cutData works for everything)w.shift option to calendarPlot to control the first day of the week and subsequent order (thanks to Giovanni Bonafè)timeAverage when expanding time seriesdplyr warnings in TheilSentype = "season" in TheilSen and smoothTrendtrajClusterdplyr bugs where some functions would fail with two typestrajClusterlinearRelation to use any arbitrary time averaging period.timeVariation when group = “season”NA factors in trajPlottrajPlot that sometimes failed to print map when groupingtrajCluster and option by.typecutDatatimeAveragelubridate for easier / faster date-time manipulationstimeProp (lattice panel.barchart is very slow)windRose (wd can be NA and ws zero i.e. calm)timeVariation when difference = TRUE for some factor levels (were in alphabetical)trendLevel; make sure strip is whitetimeVariation (vector.ws did not work)TaylorDiagram more flexible when using two groups and the second is date-basedGoogleMapsPlot when pollutant not givenGoogleMapsPlot; write to temporary file insteadaqStatsTheilSen should always give trend in units/year (would use xlab if supplied)cutData bug where quantile cuts are madescatterPlot so that factors with no data still shownTheilSen should always give trend in units/year (would use xlab if supplied)slope.text in TheilSen to allow users to add their own text i.e. not the default “units/year”aqStatsimportAURNperiod = "months" in summaryPlotmethod = "hexbin" in scatterPlotairbaseStatswindRose when all calmscatterPlotwindflow to scatterPlot and timePlot to allow wind flow plotssmoothTrendpollutionRose for option normalise to show probability by wind sector (0 to 1).timeAverage now has an option type similar to other functions. A common use would be to apply timeAverage to a data frame with multiple sites where there is a column representing site name e.g. type = "site".trajLevel and trajPlot.trend to TheilSen to control how the trend lines are drawn.calendarPlot when partial month availablecalendarPlot, don’t need to cut data firstnpoints option to trajPlot to control time spacing of dots shown on back trajectoriesstatistic = "frequency in timeAveragetimeProp due to point abovetimeVariation with type = "season" when space in pollutant namesummaryPlot - useful for shorter time seriesdplyr to speed up some of the code e.g. timeAveragex.inc and y.inc if not supplied by user in scatterPlottrajLevel frequency calculationimportMeta introduced since using dplyrangle.scale to windRose to control placement of radial scale (helps to avoid clash with wind rose paddles)timePlot when avg.time given (regression)TaylorDiagram to have group of length two. This will show all group combinations but will only differentiate them by colour/symbol according to the first grouping variable.timeVariation can now take a ylim list to control the y-limits on each individual plotmap.res to be “state” to show the US States.importKCL when date was not at beginning of the yeartimeVariation subsetsscatterPlot for surface modellingimportKCL is file of full yearaqStats when multiple pollutants selectedtimeVariation for y references line(s)corPlottrajLevel (winter period not properly calculated)timePlot, scatterPlot, timeVariation and add to smoothTrend together with full control of their properties. Note - ref.x and ref.y must now be lists; see help file for details.timeAverage for irregular time intervalspercentileRose with stat = “cpf” and non-default type (now uses single percentile based on all data, not each panel)corPlot (thanks to James Durant for the suggestion)corPlot when type = “default”pollutionRose when annotate = FALSE.scatterPlot/trajPlot when user limits suppliedsummaryPlot; some users supplied “year” resulting in incorrect statisticspollutionRose when statistic = “prop.mean” (was erroneously percentage)smoothTrend for more control over names used for plottingcorPlottimeVariation to consider median + quantiles through option ‘statistic’polarPlotimportKCL where dates were filled if two non contiguous years were chosenpolarPlot when upper is setsmoothTrend/TheilSen (was >=)polarPlot, scatterPlot, polarAnnulus when limits within data rangetimeVariation and polarAnnulus (allow any local time zone to be used; was just GMT/BST before)smoothTrend and TheilSenpercentileRose by wind direction by default.percentileRosetrajPlot bug when trajectory data was not at 3-hour intervalssmoothTrendscatterPlot surfacespolarPlot when statistic = “cpf”scatterPlot when x and/or y are on a log scaletimeAverage when requested averaging time is < original. Better handling when ‘site’ is presentpolarPlotpolarPlot to extract more source information.polarPlot to down-weight bins with few data points - alternative to min.binpolarPlottimePlot when pch supplied and group = TRUE.windRose/pollutionRose to control extent of radial limitstrajPlot and trajLevel and make default lower resolution. Make default pollutant ‘height’ (always present)calendarPlot, use first year if not supplied, add option month to allow only selected month(s)timeAveragetrajPlotscatterPlotwindRose (thanks to Philippe Barneoud from Environment Canada for pointing out the need and solution)pollutionRose when comparing two met datasets when ws bias is zerotrajLevelscatterPlotcorPlot - main did not worktimeVariation (was variable clash). Allow more flexibility when group and type are usedpolarPlot and scatterPlot (method = “level”)percentileRose.pollutionRose documentation about comparing 2 data sets (first subtracted from second)timeVariation that showed extra NA level for certain groups/typestrajLevelwindRose bug when wind direction name was not wdimportAURN when pollutant = “all” was specifiedwindRose bug where all data are missingtrajLevel for hexagonal binning of trajectory frequenciespollutionRose scaling issue, which sometimes missed the lower intervaltimeAverage data capture issue - was not always setting data to NAcalendarPlot as other functions (wrong month order in non-English locales)corPlot - see example in help filepolarPlot when radial variable was negative and smallpolarPlotpolarPlot and type is defaultwindRose when interval does not existtimeVariationwindRose for empty panels and panel labelling when data missingcalendarPlotpollutionRosepolarPlotcorPlotpolarPlots when there is insufficient data to calculate a smooth surface.timeAverage to avoid variable clash.trajLevelsummaryPlot: type = “density” broken in recent versionstimePlotimportKCL - now consistent with importAURNcalendarPlot - highlight values above/below a certain threshold.timeAverage when ws was not available but wd waswindRose when paddle = FALSEpolarPlot grid linesimportKCL that was introduced in 0.5-21smoothTrend when type = “site”windRose now gives mean ws in each panel rather than counttimePlot for more control over date format on axispollutionRose to control the width of the segmentstimeVariation to control the order of weekdays.importAURN to import new ws/wd from pre-calculated WRF data at AURN sitestimeAverage by removing date.pad.timeAverage for wind speed.NEW FUNCTION polarCluster for undertaking k-means clustering of bivariate polar plots
Remove any considerations of time zones in selectByDate. If start and end are supplied, whole days based on Date format are used to select. Now accepts start/end in the form “YYYY-mm-dd” as well as UK format e.g. dd/mm/YYYY.
polarPlot allowing variables other than “ws” to be plotted with wind direction.polarPlot/TaylorDiagrampolarPlot concentrations unless upper is suppliedpercentileRosetimeAverageTheilSen and smoothTrendtimeVariationtimeVariation to show difference between two variables with bootstrap 95% CI in the meanscatterPlot to avoid clash with x or y axis labels.corPlot, scatterPlot, smoothTrend, linearRelation, percentileRose, trajPlot, trajLevel, timeVariation, TaylorDiagram, timePlot, summaryPlot improved … handlingcorPlot added pollutant option, and openair class outputpercentileRose deal with negative dataTheilSen for all estimates for consistency. May slightly affect some p estimates.calendarPlot, kernelExceed, MannKendall and conditionalQuantile improved … handling minor update to GoogleMapsPlotwindRose and pollutionRose stat related annotationpolarPlot, ploarFreq and polarAnnulus improved … handlingimportMeta to import site meta data from air pollution networkswindRose/pollutionRose: added statistic option “abs.count”; improved scaling of segment widths; improved … handlingtrajPlot, trajLevel for importing and plotting pre-defined HYSPLIT back trajectories. These functions are under active development and are for testing purposes only!corPlot for correlation matricestimePlotwindRose/pollutionRosepercentileRosescatterPlot - not shown for some methodsscatterPlot for binning data with optional smoothing, plus other code clean-upstimePlot to allow reference lines to be addedpollutionRose to show contribution to counts and contribution to the mean. The latter is useful for displaying those wind directions that make most contribution to the overall mean. Panel mean is also now shown.importKCLtimeAverage can now expand data to shorter time periods e.g. hourly to 15-minute. This makes it more flexible to combine data sets with differing averaging times. For example, daily mean particle data can be expanded to 1-hour means and combined with an hourly meteorological data set.timeAveragetimeAverage (did not include wind speed in calculations). For most data this will make very little difference, but will be more important for low wind speeds and/or variable wind directions.smoothTrendsmoothTrend for “month” or “year” averagestimeAveragescatterPlot type = “wd” labels alignedpolarPlot, which can now consider “mean”, “median”, “max” (maximum), “frequency”. “stdev” (standard deviation) or “weighted.mean” in a similar way to polarFreqsmoothTrend/MannKendall; particularly when there are multiple sitestimePlot would fail with percentiles)smoothTrend, MannKendall, timePlot and scatterPlotimportKCL, added site classification to the help file and ensured GMT is exported.smoothTrend and MannKendall to allow flexible y-scales.percentileRose’ for flexible plotting of percentiles by wind directionselectRunning for selecting run lengths of a numeric variable above a certain threshold.polarPlot to maximise plotting areawindRose and percentileRose; new option grid.line to control radial axes.summaryPlot and timePlotwindRose and pollutionRose on radial scalecalendarPlot main/quickText handling.calendarPlot first day of month error.summaryPlot site ordering when more than one site.scatterPlot (plot.type, lwd, and lty) making it possible to add points and/or linesAdded new function conditionalQuantile for model
evaluation purposes.Fixed `importAURN` to account for a change in web domain
address at AEA.