diff --git a/Documentation/block/bfq-iosched.txt b/Documentation/block/bfq-iosched.txt index 05e2822..6aa5a88 100644 --- a/Documentation/block/bfq-iosched.txt +++ b/Documentation/block/bfq-iosched.txt @@ -11,21 +11,32 @@ controllers), BFQ's main features are: groups (switching back to time distribution when needed to keep throughput high). +If bfq-mq patches have been applied, then the following three +instances of BFQ are available (otherwise only the first instance): +- bfq: mainline version of BFQ, for blk-mq +- bfq-mq: development version of BFQ for blk-mq; this version contains + also all latest features and fixes not yet landed in mainline, plus many + safety checks +- bfq-sq: BFQ for legacy blk; also this version contains latest features + and fixes, as well as safety checks + In its default configuration, BFQ privileges latency over throughput. So, when needed for achieving a lower latency, BFQ builds schedules that may lead to a lower throughput. If your main or only goal, for a given device, is to achieve the maximum-possible throughput at all times, then do switch off all low-latency heuristics -for that device, by setting low_latency to 0. Full details in Section 3. +for that device, by setting low_latency to 0. See Section 3 for +details on how to configure BFQ for the desired tradeoff between +latency and throughput, or on how to maximize throughput. On average CPUs, the current version of BFQ can handle devices performing at most ~30K IOPS; at most ~50 KIOPS on faster CPUs. As a reference, 30-50 KIOPS correspond to very high bandwidths with sequential I/O (e.g., 8-12 GB/s if I/O requests are 256 KB large), and -to 120-200 MB/s with 4KB random I/O. BFQ has not yet been tested on -multi-queue devices. +to 120-200 MB/s with 4KB random I/O. BFQ is currently being tested on +multi-queue devices too. -The table of contents follow. Impatients can just jump to Section 3. +The table of contents follows. Impatients can just jump to Section 3. CONTENTS @@ -33,7 +44,7 @@ CONTENTS 1-1 Personal systems 1-2 Server systems 2. How does BFQ work? -3. What are BFQ's tunable? +3. What are BFQ's tunables and how to properly configure BFQ? 4. BFQ group scheduling 4-1 Service guarantees provided 4-2 Interface @@ -145,19 +156,28 @@ plus a lot of code, are borrowed from CFQ. contrast, BFQ may idle the device for a short time interval, giving the process the chance to go on being served if it issues a new request in time. Device idling typically boosts the - throughput on rotational devices, if processes do synchronous - and sequential I/O. In addition, under BFQ, device idling is - also instrumental in guaranteeing the desired throughput - fraction to processes issuing sync requests (see the description - of the slice_idle tunable in this document, or [1, 2], for more - details). + throughput on rotational devices and on non-queueing flash-based + devices, if processes do synchronous and sequential I/O. In + addition, under BFQ, device idling is also instrumental in + guaranteeing the desired throughput fraction to processes + issuing sync requests (see the description of the slice_idle + tunable in this document, or [1, 2], for more details). - With respect to idling for service guarantees, if several processes are competing for the device at the same time, but - all processes (and groups, after the following commit) have - the same weight, then BFQ guarantees the expected throughput - distribution without ever idling the device. Throughput is - thus as high as possible in this common scenario. + all processes and groups have the same weight, then BFQ + guarantees the expected throughput distribution without ever + idling the device. Throughput is thus as high as possible in + this common scenario. + + - On flash-based storage with internal queueing of commands + (typically NCQ), device idling happens to be always detrimental + for throughput. So, with these devices, BFQ performs idling + only when strictly needed for service guarantees, i.e., for + guaranteeing low latency or fairness. In these cases, overall + throughput may be sub-optimal. No solution currently exists to + provide both strong service guarantees and optimal throughput + on devices with internal queueing. - If low-latency mode is enabled (default configuration), BFQ executes some special heuristics to detect interactive and soft @@ -191,10 +211,7 @@ plus a lot of code, are borrowed from CFQ. - Queues are scheduled according to a variant of WF2Q+, named B-WF2Q+, and implemented using an augmented rb-tree to preserve an O(log N) overall complexity. See [2] for more details. B-WF2Q+ is - also ready for hierarchical scheduling. However, for a cleaner - logical breakdown, the code that enables and completes - hierarchical support is provided in the next commit, which focuses - exactly on this feature. + also ready for hierarchical scheduling, details in Section 4. - B-WF2Q+ guarantees a tight deviation with respect to an ideal, perfectly fair, and smooth service. In particular, B-WF2Q+ @@ -249,13 +266,24 @@ plus a lot of code, are borrowed from CFQ. the Idle class, to prevent it from starving. -3. What are BFQ's tunable? -========================== +3. What are BFQ's tunables and how to properly configure BFQ? +============================================================= -The tunables back_seek-max, back_seek_penalty, fifo_expire_async and -fifo_expire_sync below are the same as in CFQ. Their description is -just copied from that for CFQ. Some considerations in the description -of slice_idle are copied from CFQ too. +Most BFQ tunables affect service guarantees (basically latency and +fairness) and throughput. For full details on how to choose the +desired tradeoff between service guarantees and throughput, see the +parameters slice_idle, strict_guarantees and low_latency. For details +on how to maximise throughput, see slice_idle, timeout_sync and +max_budget. The other performance-related parameters have been +inherited from, and have been preserved mostly for compatibility with +CFQ. So far, no performance improvement has been reported after +changing the latter parameters in BFQ. + +In particular, the tunables back_seek-max, back_seek_penalty, +fifo_expire_async and fifo_expire_sync below are the same as in +CFQ. Their description is just copied from that for CFQ. Some +considerations in the description of slice_idle are copied from CFQ +too. per-process ioprio and weight ----------------------------- @@ -285,15 +313,17 @@ number of seeks and see improved throughput. Setting slice_idle to 0 will remove all the idling on queues and one should see an overall improved throughput on faster storage devices -like multiple SATA/SAS disks in hardware RAID configuration. +like multiple SATA/SAS disks in hardware RAID configuration, as well +as flash-based storage with internal command queueing (and +parallelism). So depending on storage and workload, it might be useful to set slice_idle=0. In general for SATA/SAS disks and software RAID of SATA/SAS disks keeping slice_idle enabled should be useful. For any configurations where there are multiple spindles behind single LUN -(Host based hardware RAID controller or for storage arrays), setting -slice_idle=0 might end up in better throughput and acceptable -latencies. +(Host based hardware RAID controller or for storage arrays), or with +flash-based fast storage, setting slice_idle=0 might end up in better +throughput and acceptable latencies. Idling is however necessary to have service guarantees enforced in case of differentiated weights or differentiated I/O-request lengths. @@ -312,13 +342,14 @@ There is an important flipside for idling: apart from the above cases where it is beneficial also for throughput, idling can severely impact throughput. One important case is random workload. Because of this issue, BFQ tends to avoid idling as much as possible, when it is not -beneficial also for throughput. As a consequence of this behavior, and -of further issues described for the strict_guarantees tunable, -short-term service guarantees may be occasionally violated. And, in -some cases, these guarantees may be more important than guaranteeing -maximum throughput. For example, in video playing/streaming, a very -low drop rate may be more important than maximum throughput. In these -cases, consider setting the strict_guarantees parameter. +beneficial also for throughput (as detailed in Section 2). As a +consequence of this behavior, and of further issues described for the +strict_guarantees tunable, short-term service guarantees may be +occasionally violated. And, in some cases, these guarantees may be +more important than guaranteeing maximum throughput. For example, in +video playing/streaming, a very low drop rate may be more important +than maximum throughput. In these cases, consider setting the +strict_guarantees parameter. strict_guarantees ----------------- @@ -420,6 +451,13 @@ The default value is 0, which enables auto-tuning: BFQ sets max_budget to the maximum number of sectors that can be served during timeout_sync, according to the estimated peak rate. +For specific devices, some users have occasionally reported to have +reached a higher throughput by setting max_budget explicitly, i.e., by +setting max_budget to a higher value than 0. In particular, they have +set max_budget to higher values than those to which BFQ would have set +it with auto-tuning. An alternative way to achieve this goal is to +just increase the value of timeout_sync, leaving max_budget equal to 0. + weights ------- @@ -510,10 +548,12 @@ To get proportional sharing of bandwidth with BFQ for a given device, BFQ must of course be the active scheduler for that device. Within each group directory, the names of the files associated with -BFQ-specific cgroup parameters and stats begin with the "bfq." -prefix. So, with cgroups-v1 or cgroups-v2, the full prefix for -BFQ-specific files is "blkio.bfq." or "io.bfq." For example, the group -parameter to set the weight of a group with BFQ is blkio.bfq.weight +BFQ-specific cgroup parameters and stats begin with the "bfq.", +"bfq-sq." or "bfq-mq." prefix, depending on which instance of bfq you +want to use. So, with cgroups-v1 or cgroups-v2, the full prefix for +BFQ-specific files is "blkio.bfqX." or "io.bfqX.", where X can be "" +(i.e., null string), "-sq" or "-mq". For example, the group parameter +to set the weight of a group with the mainline BFQ is blkio.bfq.weight or io.bfq.weight. Parameters to set @@ -521,7 +561,7 @@ Parameters to set For each group, there is only the following parameter to set. -weight (namely blkio.bfq.weight or io.bfq-weight): the weight of the +weight (namely blkio.bfqX.weight or io.bfqX.weight): the weight of the group inside its parent. Available values: 1..10000 (default 100). The linear mapping between ioprio and weights, described at the beginning of the tunable section, is still valid, but all weights higher than diff --git b/Documentation/tp_smapi.txt b/Documentation/tp_smapi.txt new file mode 100644 index 0000000..d037301 --- /dev/null +++ b/Documentation/tp_smapi.txt @@ -0,0 +1,267 @@ +tp_smapi version 0.40 +IBM ThinkPad hardware functions driver + +Author: Shem Multinymous +Project: http://sourceforge.net/projects/tpctl +Wiki: http://thinkwiki.org/wiki/tp_smapi +List: linux-thinkpad@linux-thinkpad.org + (http://mailman.linux-thinkpad.org/mailman/listinfo/linux-thinkpad) + +Description +----------- + +ThinkPad laptops include a proprietary interface called SMAPI BIOS +(System Management Application Program Interface) which provides some +hardware control functionality that is not accessible by other means. + +This driver exposes some features of the SMAPI BIOS through a sysfs +interface. It is suitable for newer models, on which SMAPI is invoked +through IO port writes. Older models use a different SMAPI interface; +for those, try the "thinkpad" module from the "tpctl" package. + +WARNING: +This driver uses undocumented features and direct hardware access. +It thus cannot be guaranteed to work, and may cause arbitrary damage +(especially on models it wasn't tested on). + + +Module parameters +----------------- + +thinkpad_ec module: + force_io=1 lets thinkpad_ec load on some recent ThinkPad models + (e.g., T400 and T500) whose BIOS's ACPI DSDT reserves the ports we need. +tp_smapi module: + debug=1 enables verbose dmesg output. + + +Usage +----- + +Control of battery charging thresholds (in percents of current full charge +capacity): + +# echo 40 > /sys/devices/platform/smapi/BAT0/start_charge_thresh +# echo 70 > /sys/devices/platform/smapi/BAT0/stop_charge_thresh +# cat /sys/devices/platform/smapi/BAT0/*_charge_thresh + + (This is useful since Li-Ion batteries wear out much faster at very + high or low charge levels. The driver will also keeps the thresholds + across suspend-to-disk with AC disconnected; this isn't done + automatically by the hardware.) + +Inhibiting battery charging for 17 minutes (overrides thresholds): + +# echo 17 > /sys/devices/platform/smapi/BAT0/inhibit_charge_minutes +# echo 0 > /sys/devices/platform/smapi/BAT0/inhibit_charge_minutes # stop +# cat /sys/devices/platform/smapi/BAT0/inhibit_charge_minutes + + (This can be used to control which battery is charged when using an + Ultrabay battery.) + +Forcing battery discharging even if AC power available: + +# echo 1 > /sys/devices/platform/smapi/BAT0/force_discharge # start discharge +# echo 0 > /sys/devices/platform/smapi/BAT0/force_discharge # stop discharge +# cat /sys/devices/platform/smapi/BAT0/force_discharge + + (When AC is connected, forced discharging will automatically stop + when battery is fully depleted -- this is useful for calibration. + Also, this attribute can be used to control which battery is discharged + when both a system battery and an Ultrabay battery are connected.) + +Misc read-only battery status attributes (see note about HDAPS below): + +/sys/devices/platform/smapi/BAT0/installed # 0 or 1 +/sys/devices/platform/smapi/BAT0/state # idle/charging/discharging +/sys/devices/platform/smapi/BAT0/cycle_count # integer counter +/sys/devices/platform/smapi/BAT0/current_now # instantaneous current +/sys/devices/platform/smapi/BAT0/current_avg # last minute average +/sys/devices/platform/smapi/BAT0/power_now # instantaneous power +/sys/devices/platform/smapi/BAT0/power_avg # last minute average +/sys/devices/platform/smapi/BAT0/last_full_capacity # in mWh +/sys/devices/platform/smapi/BAT0/remaining_percent # remaining percent of energy (set by calibration) +/sys/devices/platform/smapi/BAT0/remaining_percent_error # error range of remaing_percent (not reset by calibration) +/sys/devices/platform/smapi/BAT0/remaining_running_time # in minutes, by last minute average power +/sys/devices/platform/smapi/BAT0/remaining_running_time_now # in minutes, by instantenous power +/sys/devices/platform/smapi/BAT0/remaining_charging_time # in minutes +/sys/devices/platform/smapi/BAT0/remaining_capacity # in mWh +/sys/devices/platform/smapi/BAT0/design_capacity # in mWh +/sys/devices/platform/smapi/BAT0/voltage # in mV +/sys/devices/platform/smapi/BAT0/design_voltage # in mV +/sys/devices/platform/smapi/BAT0/charging_max_current # max charging current +/sys/devices/platform/smapi/BAT0/charging_max_voltage # max charging voltage +/sys/devices/platform/smapi/BAT0/group{0,1,2,3}_voltage # see below +/sys/devices/platform/smapi/BAT0/manufacturer # string +/sys/devices/platform/smapi/BAT0/model # string +/sys/devices/platform/smapi/BAT0/barcoding # string +/sys/devices/platform/smapi/BAT0/chemistry # string +/sys/devices/platform/smapi/BAT0/serial # integer +/sys/devices/platform/smapi/BAT0/manufacture_date # YYYY-MM-DD +/sys/devices/platform/smapi/BAT0/first_use_date # YYYY-MM-DD +/sys/devices/platform/smapi/BAT0/temperature # in milli-Celsius +/sys/devices/platform/smapi/BAT0/dump # see below +/sys/devices/platform/smapi/ac_connected # 0 or 1 + +The BAT0/group{0,1,2,3}_voltage attribute refers to the separate cell groups +in each battery. For example, on the ThinkPad 600, X3x, T4x and R5x models, +the battery contains 3 cell groups in series, where each group consisting of 2 +or 3 cells connected in parallel. The voltage of each group is given by these +attributes, and their sum (roughly) equals the "voltage" attribute. +(The effective performance of the battery is determined by the weakest group, +i.e., the one those voltage changes most rapidly during dis/charging.) + +The "BAT0/dump" attribute gives a a hex dump of the raw status data, which +contains additional data now in the above (if you can figure it out). Some +unused values are autodetected and replaced by "--": + +In all of the above, replace BAT0 with BAT1 to address the 2nd battery (e.g. +in the UltraBay). + + +Raw SMAPI calls: + +/sys/devices/platform/smapi/smapi_request +This performs raw SMAPI calls. It uses a bad interface that cannot handle +multiple simultaneous access. Don't touch it, it's for development only. +If you did touch it, you would so something like +# echo '211a 100 0 0' > /sys/devices/platform/smapi/smapi_request +# cat /sys/devices/platform/smapi/smapi_request +and notice that in the output "211a 34b b2 0 0 0 'OK'", the "4b" in the 2nd +value, converted to decimal is 75: the current charge stop threshold. + + +Model-specific status +--------------------- + +Works (at least partially) on the following ThinkPad model: +* A30 +* G41 +* R40, R50p, R51, R52 +* T23, T40, T40p, T41, T41p, T42, T42p, T43, T43p, T60 +* X24, X31, X32, X40, X41, X60 +* Z60t, Z61m + +Not all functions are available on all models; for detailed status, see: + http://thinkwiki.org/wiki/tp_smapi + +Please report success/failure by e-mail or on the Wiki. +If you get a "not implemented" or "not supported" message, your laptop +probably just can't do that (at least not via the SMAPI BIOS). +For negative reports, follow the bug reporting guidelines below. +If you send me the necessary technical data (i.e., SMAPI function +interfaces), I will support additional models. + + +Additional HDAPS features +------------------------- + +The modified hdaps driver has several improvements on the one in mainline +(beyond resolving the conflict with thinkpad_ec and tp_smapi): + +- Fixes reliability and improves support for recent ThinkPad models + (especially *60 and newer). Unlike the mainline driver, the modified hdaps + correctly follows the Embedded Controller communication protocol. + +- Extends the "invert" parameter to cover all possible axis orientations. + The possible values are as follows. + Let X,Y denote the hardware readouts. + Let R denote the laptop's roll (tilt left/right). + Let P denote the laptop's pitch (tilt forward/backward). + invert=0: R= X P= Y (same as mainline) + invert=1: R=-X P=-Y (same as mainline) + invert=2: R=-X P= Y (new) + invert=3: R= X P=-Y (new) + invert=4: R= Y P= X (new) + invert=5: R=-Y P=-X (new) + invert=6: R=-Y P= X (new) + invert=7: R= Y P=-X (new) + It's probably easiest to just try all 8 possibilities and see which yields + correct results (e.g., in the hdaps-gl visualisation). + +- Adds a whitelist which automatically sets the correct axis orientation for + some models. If the value for your model is wrong or missing, you can override + it using the "invert" parameter. Please also update the tables at + http://www.thinkwiki.org/wiki/tp_smapi and + http://www.thinkwiki.org/wiki/List_of_DMI_IDs + and submit a patch for the whitelist in hdaps.c. + +- Provides new attributes: + /sys/devices/platform/hdaps/sampling_rate: + This determines the frequency at which the host queries the embedded + controller for accelerometer data (and informs the hdaps input devices). + Default=50. + /sys/devices/platform/hdaps/oversampling_ratio: + When set to X, the embedded controller is told to do physical accelerometer + measurements at a rate that is X times higher than the rate at which + the driver reads those measurements (i.e., X*sampling_rate). This + makes the readouts from the embedded controller more fresh, and is also + useful for the running average filter (see next). Default=5 + /sys/devices/platform/hdaps/running_avg_filter_order: + When set to X, reported readouts will be the average of the last X physical + accelerometer measurements. Current firmware allows 1<=X<=8. Setting to a + high value decreases readout fluctuations. The averaging is handled by the + embedded controller, so no CPU resources are used. Higher values make the + readouts smoother, since it averages out both sensor noise (good) and abrupt + changes (bad). Default=2. + +- Provides a second input device, which publishes the raw accelerometer + measurements (without the fuzzing needed for joystick emulation). This input + device can be matched by a udev rule such as the following (all on one line): + KERNEL=="event[0-9]*", ATTRS{phys}=="hdaps/input1", + ATTRS{modalias}=="input:b0019v1014p5054e4801-*", + SYMLINK+="input/hdaps/accelerometer-event + +A new version of the hdapsd userspace daemon, which uses the input device +interface instead of polling sysfs, is available seprately. Using this reduces +the total interrupts per second generated by hdaps+hdapsd (on tickless kernels) +to 50, down from a value that fluctuates between 50 and 100. Set the +sampling_rate sysfs attribute to a lower value to further reduce interrupts, +at the expense of response latency. + +Licensing note: all my changes to the HDAPS driver are licensed under the +GPL version 2 or, at your option and to the extent allowed by derivation from +prior works, any later version. My version of hdaps is derived work from the +mainline version, which at the time of writing is available only under +GPL version 2. + +Bug reporting +------------- + +Mail . Please include: +* Details about your model, +* Relevant "dmesg" output. Make sure thinkpad_ec and tp_smapi are loaded with + the "debug=1" parameter (e.g., use "make load HDAPS=1 DEBUG=1"). +* Output of "dmidecode | grep -C5 Product" +* Does the failed functionality works under Windows? + + +More about SMAPI +---------------- + +For hints about what may be possible via the SMAPI BIOS and how, see: + +* IBM Technical Reference Manual for the ThinkPad 770 + (http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=PFAN-3TUQQD) +* Exported symbols in PWRMGRIF.DLL or TPPWRW32.DLL (e.g., use "objdump -x"). +* drivers/char/mwave/smapi.c in the Linux kernel tree.* +* The "thinkpad" SMAPI module (http://tpctl.sourceforge.net). +* The SMAPI_* constants in tp_smapi.c. + +Note that in the above Technical Reference and in the "thinkpad" module, +SMAPI is invoked through a function call to some physical address. However, +the interface used by tp_smapi and the above mwave drive, and apparently +required by newer ThinkPad, is different: you set the parameters up in the +CPU's registers and write to ports 0xB2 (the APM control port) and 0x4F; this +triggers an SMI (System Management Interrupt), causing the CPU to enter +SMM (System Management Mode) and run the BIOS firmware; the results are +returned in the CPU's registers. It is not clear what is the relation between +the two variants of SMAPI, though the assignment of error codes seems to be +similar. + +In addition, the embedded controller on ThinkPad laptops has a non-standard +interface at IO ports 0x1600-0x161F (mapped to LCP channel 3 of the H8S chip). +The interface provides various system management services (currently known: +battery information and accelerometer readouts). For more information see the +thinkpad_ec module and the H8S hardware documentation: +http://documentation.renesas.com/eng/products/mpumcu/rej09b0300_2140bhm.pdf diff --git a/Makefile b/Makefile index 187dc00..d826a86 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ VERSION = 4 PATCHLEVEL = 13 SUBLEVEL = 5 -EXTRAVERSION = -gnu +EXTRAVERSION = -pck NAME = Fearless Coyote # *DOCUMENTATION* @@ -638,12 +638,16 @@ ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE KBUILD_CFLAGS += $(call cc-option,-Oz,-Os) KBUILD_CFLAGS += $(call cc-disable-warning,maybe-uninitialized,) else +ifdef CONFIG_CC_OPTIMIZE_HARDER +KBUILD_CFLAGS += -O3 $(call cc-disable-warning,maybe-uninitialized,) +else ifdef CONFIG_PROFILE_ALL_BRANCHES KBUILD_CFLAGS += -O2 $(call cc-disable-warning,maybe-uninitialized,) else KBUILD_CFLAGS += -O2 endif endif +endif KBUILD_CFLAGS += $(call cc-ifversion, -lt, 0409, \ $(call cc-disable-warning,maybe-uninitialized,)) diff --git a/arch/x86/Kconfig.cpu b/arch/x86/Kconfig.cpu index 3ba5ff2..a0d1e7b 100644 --- a/arch/x86/Kconfig.cpu +++ b/arch/x86/Kconfig.cpu @@ -115,6 +115,7 @@ config MPENTIUMM config MPENTIUM4 bool "Pentium-4/Celeron(P4-based)/Pentium-4 M/older Xeon" depends on X86_32 + select X86_P6_NOP ---help--- Select this for Intel Pentium 4 chips. This includes the Pentium 4, Pentium D, P4-based Celeron and Xeon, and @@ -147,9 +148,8 @@ config MPENTIUM4 -Paxville -Dempsey - config MK6 - bool "K6/K6-II/K6-III" + bool "AMD K6/K6-II/K6-III" depends on X86_32 ---help--- Select this for an AMD K6-family processor. Enables use of @@ -157,7 +157,7 @@ config MK6 flags to GCC. config MK7 - bool "Athlon/Duron/K7" + bool "AMD Athlon/Duron/K7" depends on X86_32 ---help--- Select this for an AMD Athlon K7-family processor. Enables use of @@ -165,12 +165,83 @@ config MK7 flags to GCC. config MK8 - bool "Opteron/Athlon64/Hammer/K8" + bool "AMD Opteron/Athlon64/Hammer/K8" ---help--- Select this for an AMD Opteron or Athlon64 Hammer-family processor. Enables use of some extended instructions, and passes appropriate optimization flags to GCC. +config MK8SSE3 + bool "AMD Opteron/Athlon64/Hammer/K8 with SSE3" + ---help--- + Select this for improved AMD Opteron or Athlon64 Hammer-family processors. + Enables use of some extended instructions, and passes appropriate + optimization flags to GCC. + +config MK10 + bool "AMD 61xx/7x50/PhenomX3/X4/II/K10" + ---help--- + Select this for an AMD 61xx Eight-Core Magny-Cours, Athlon X2 7x50, + Phenom X3/X4/II, Athlon II X2/X3/X4, or Turion II-family processor. + Enables use of some extended instructions, and passes appropriate + optimization flags to GCC. + +config MBARCELONA + bool "AMD Barcelona" + ---help--- + Select this for AMD Family 10h Barcelona processors. + + Enables -march=barcelona + +config MBOBCAT + bool "AMD Bobcat" + ---help--- + Select this for AMD Family 14h Bobcat processors. + + Enables -march=btver1 + +config MJAGUAR + bool "AMD Jaguar" + ---help--- + Select this for AMD Family 16h Jaguar processors. + + Enables -march=btver2 + +config MBULLDOZER + bool "AMD Bulldozer" + ---help--- + Select this for AMD Family 15h Bulldozer processors. + + Enables -march=bdver1 + +config MPILEDRIVER + bool "AMD Piledriver" + ---help--- + Select this for AMD Family 15h Piledriver processors. + + Enables -march=bdver2 + +config MSTEAMROLLER + bool "AMD Steamroller" + ---help--- + Select this for AMD Family 15h Steamroller processors. + + Enables -march=bdver3 + +config MEXCAVATOR + bool "AMD Excavator" + ---help--- + Select this for AMD Family 15h Excavator processors. + + Enables -march=bdver4 + +config MPCK + bool "AMD Zen" + ---help--- + Select this for AMD Family 17h Zen processors. + + Enables -march=znver1 + config MCRUSOE bool "Crusoe" depends on X86_32 @@ -252,6 +323,7 @@ config MVIAC7 config MPSC bool "Intel P4 / older Netburst based Xeon" + select X86_P6_NOP depends on X86_64 ---help--- Optimize for Intel Pentium 4, Pentium D and older Nocona/Dempsey @@ -261,8 +333,19 @@ config MPSC using the cpu family field in /proc/cpuinfo. Family 15 is an older Xeon, Family 6 a newer one. +config MATOM + bool "Intel Atom" + select X86_P6_NOP + ---help--- + + Select this for the Intel Atom platform. Intel Atom CPUs have an + in-order pipelining architecture and thus can benefit from + accordingly optimized code. Use a recent GCC with specific Atom + support in order to fully benefit from selecting this option. + config MCORE2 - bool "Core 2/newer Xeon" + bool "Intel Core 2" + select X86_P6_NOP ---help--- Select this for Intel Core 2 and newer Core 2 Xeons (Xeon 51xx and @@ -270,14 +353,79 @@ config MCORE2 family in /proc/cpuinfo. Newer ones have 6 and older ones 15 (not a typo) -config MATOM - bool "Intel Atom" + Enables -march=core2 + +config MNEHALEM + bool "Intel Nehalem" + select X86_P6_NOP ---help--- - Select this for the Intel Atom platform. Intel Atom CPUs have an - in-order pipelining architecture and thus can benefit from - accordingly optimized code. Use a recent GCC with specific Atom - support in order to fully benefit from selecting this option. + Select this for 1st Gen Core processors in the Nehalem family. + + Enables -march=nehalem + +config MWESTMERE + bool "Intel Westmere" + select X86_P6_NOP + ---help--- + + Select this for the Intel Westmere formerly Nehalem-C family. + + Enables -march=westmere + +config MSILVERMONT + bool "Intel Silvermont" + select X86_P6_NOP + ---help--- + + Select this for the Intel Silvermont platform. + + Enables -march=silvermont + +config MSANDYBRIDGE + bool "Intel Sandy Bridge" + select X86_P6_NOP + ---help--- + + Select this for 2nd Gen Core processors in the Sandy Bridge family. + + Enables -march=sandybridge + +config MIVYBRIDGE + bool "Intel Ivy Bridge" + select X86_P6_NOP + ---help--- + + Select this for 3rd Gen Core processors in the Ivy Bridge family. + + Enables -march=ivybridge + +config MHASWELL + bool "Intel Haswell" + select X86_P6_NOP + ---help--- + + Select this for 4th Gen Core processors in the Haswell family. + + Enables -march=haswell + +config MBROADWELL + bool "Intel Broadwell" + select X86_P6_NOP + ---help--- + + Select this for 5th Gen Core processors in the Broadwell family. + + Enables -march=broadwell + +config MSKYLAKE + bool "Intel Skylake" + select X86_P6_NOP + ---help--- + + Select this for 6th Gen Core processors in the Skylake family. + + Enables -march=skylake config GENERIC_CPU bool "Generic-x86-64" @@ -286,6 +434,19 @@ config GENERIC_CPU Generic x86-64 CPU. Run equally well on all x86-64 CPUs. +config MNATIVE + bool "Native optimizations autodetected by GCC" + ---help--- + + GCC 4.2 and above support -march=native, which automatically detects + the optimum settings to use based on your processor. -march=native + also detects and applies additional settings beyond -march specific + to your CPU, (eg. -msse4). Unless you have a specific reason not to + (e.g. distcc cross-compiling), you should probably be using + -march=native rather than anything listed below. + + Enables -march=native + endchoice config X86_GENERIC @@ -310,7 +471,7 @@ config X86_INTERNODE_CACHE_SHIFT config X86_L1_CACHE_SHIFT int default "7" if MPENTIUM4 || MPSC - default "6" if MK7 || MK8 || MPENTIUMM || MCORE2 || MATOM || MVIAC7 || X86_GENERIC || GENERIC_CPU + default "6" if MK7 || MK8 || MK8SSE3 || MK10 || MBARCELONA || MBOBCAT || MBULLDOZER || MPILEDRIVER || MSTEAMROLLER || MEXCAVATOR || MPCK || MJAGUAR || MPENTIUMM || MCORE2 || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MNATIVE || MATOM || MVIAC7 || X86_GENERIC || GENERIC_CPU default "4" if MELAN || M486 || MGEODEGX1 default "5" if MWINCHIP3D || MWINCHIPC6 || MCRUSOE || MEFFICEON || MCYRIXIII || MK6 || MPENTIUMIII || MPENTIUMII || M686 || M586MMX || M586TSC || M586 || MVIAC3_2 || MGEODE_LX @@ -341,45 +502,46 @@ config X86_ALIGNMENT_16 config X86_INTEL_USERCOPY def_bool y - depends on MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M586MMX || X86_GENERIC || MK8 || MK7 || MEFFICEON || MCORE2 + depends on MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M586MMX || X86_GENERIC || MK8 || MK8SSE3 || MK7 || MEFFICEON || MCORE2 || MK10 || MBARCELONA || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MNATIVE config X86_USE_PPRO_CHECKSUM def_bool y - depends on MWINCHIP3D || MWINCHIPC6 || MCYRIXIII || MK7 || MK6 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MK8 || MVIAC3_2 || MVIAC7 || MEFFICEON || MGEODE_LX || MCORE2 || MATOM + depends on MWINCHIP3D || MWINCHIPC6 || MCYRIXIII || MK7 || MK6 || MK10 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MK8 || MK8SSE3 || MVIAC3_2 || MVIAC7 || MEFFICEON || MGEODE_LX || MCORE2 || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MATOM || MNATIVE config X86_USE_3DNOW def_bool y depends on (MCYRIXIII || MK7 || MGEODE_LX) && !UML -# -# P6_NOPs are a relatively minor optimization that require a family >= -# 6 processor, except that it is broken on certain VIA chips. -# Furthermore, AMD chips prefer a totally different sequence of NOPs -# (which work on all CPUs). In addition, it looks like Virtual PC -# does not understand them. -# -# As a result, disallow these if we're not compiling for X86_64 (these -# NOPs do work on all x86-64 capable chips); the list of processors in -# the right-hand clause are the cores that benefit from this optimization. -# config X86_P6_NOP - def_bool y - depends on X86_64 - depends on (MCORE2 || MPENTIUM4 || MPSC) + default n + bool "Support for P6_NOPs on Intel chips" + depends on (MCORE2 || MPENTIUM4 || MPSC || MATOM || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MNATIVE) + ---help--- + P6_NOPs are a relatively minor optimization that require a family >= + 6 processor, except that it is broken on certain VIA chips. + Furthermore, AMD chips prefer a totally different sequence of NOPs + (which work on all CPUs). In addition, it looks like Virtual PC + does not understand them. + + As a result, disallow these if we're not compiling for X86_64 (these + NOPs do work on all x86-64 capable chips); the list of processors in + the right-hand clause are the cores that benefit from this optimization. + + Say Y if you have Intel CPU newer than Pentium Pro, N otherwise. config X86_TSC def_bool y - depends on (MWINCHIP3D || MCRUSOE || MEFFICEON || MCYRIXIII || MK7 || MK6 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || M586MMX || M586TSC || MK8 || MVIAC3_2 || MVIAC7 || MGEODEGX1 || MGEODE_LX || MCORE2 || MATOM) || X86_64 + depends on (MWINCHIP3D || MCRUSOE || MEFFICEON || MCYRIXIII || MK7 || MK6 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || M586MMX || M586TSC || MK8 || MK8SSE3 || MVIAC3_2 || MVIAC7 || MGEODEGX1 || MGEODE_LX || MCORE2 || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MNATIVE || MATOM) || X86_64 config X86_CMPXCHG64 def_bool y - depends on X86_PAE || X86_64 || MCORE2 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MATOM + depends on X86_PAE || X86_64 || MCORE2 || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MATOM || MNATIVE # this should be set for all -march=.. options where the compiler # generates cmov. config X86_CMOV def_bool y - depends on (MK8 || MK7 || MCORE2 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MVIAC3_2 || MVIAC7 || MCRUSOE || MEFFICEON || X86_64 || MATOM || MGEODE_LX) + depends on (MK8 || MK8SSE3 || MK10 || MBARCELONA || MBOBCAT || MBULLDOZER || MPILEDRIVER || MSTEAMROLLER || MEXCAVATOR || MPCK || MJAGUAR || MK7 || MCORE2 || MNEHALEM || MWESTMERE || MSILVERMONT || MSANDYBRIDGE || MIVYBRIDGE || MHASWELL || MBROADWELL || MSKYLAKE || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || MVIAC3_2 || MVIAC7 || MCRUSOE || MEFFICEON || X86_64 || MNATIVE || MATOM || MGEODE_LX) config X86_MINIMUM_CPU_FAMILY int diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 1e902f9..bbdc355 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -121,13 +121,40 @@ else KBUILD_CFLAGS += $(call cc-option,-mskip-rax-setup) # FIXME - should be integrated in Makefile.cpu (Makefile_32.cpu) + cflags-$(CONFIG_MNATIVE) += $(call cc-option,-march=native) cflags-$(CONFIG_MK8) += $(call cc-option,-march=k8) + cflags-$(CONFIG_MK8SSE3) += $(call cc-option,-march=k8-sse3,-mtune=k8) + cflags-$(CONFIG_MK10) += $(call cc-option,-march=amdfam10) + cflags-$(CONFIG_MBARCELONA) += $(call cc-option,-march=barcelona) + cflags-$(CONFIG_MBOBCAT) += $(call cc-option,-march=btver1) + cflags-$(CONFIG_MJAGUAR) += $(call cc-option,-march=btver2) + cflags-$(CONFIG_MBULLDOZER) += $(call cc-option,-march=bdver1) + cflags-$(CONFIG_MPILEDRIVER) += $(call cc-option,-march=bdver2) + cflags-$(CONFIG_MSTEAMROLLER) += $(call cc-option,-march=bdver3) + cflags-$(CONFIG_MEXCAVATOR) += $(call cc-option,-march=bdver4) + cflags-$(CONFIG_MPCK) += $(call cc-option,-march=znver1) cflags-$(CONFIG_MPSC) += $(call cc-option,-march=nocona) cflags-$(CONFIG_MCORE2) += \ - $(call cc-option,-march=core2,$(call cc-option,-mtune=generic)) - cflags-$(CONFIG_MATOM) += $(call cc-option,-march=atom) \ - $(call cc-option,-mtune=atom,$(call cc-option,-mtune=generic)) + $(call cc-option,-march=core2,$(call cc-option,-mtune=core2)) + cflags-$(CONFIG_MNEHALEM) += \ + $(call cc-option,-march=nehalem,$(call cc-option,-mtune=nehalem)) + cflags-$(CONFIG_MWESTMERE) += \ + $(call cc-option,-march=westmere,$(call cc-option,-mtune=westmere)) + cflags-$(CONFIG_MSILVERMONT) += \ + $(call cc-option,-march=silvermont,$(call cc-option,-mtune=silvermont)) + cflags-$(CONFIG_MSANDYBRIDGE) += \ + $(call cc-option,-march=sandybridge,$(call cc-option,-mtune=sandybridge)) + cflags-$(CONFIG_MIVYBRIDGE) += \ + $(call cc-option,-march=ivybridge,$(call cc-option,-mtune=ivybridge)) + cflags-$(CONFIG_MHASWELL) += \ + $(call cc-option,-march=haswell,$(call cc-option,-mtune=haswell)) + cflags-$(CONFIG_MBROADWELL) += \ + $(call cc-option,-march=broadwell,$(call cc-option,-mtune=broadwell)) + cflags-$(CONFIG_MSKYLAKE) += \ + $(call cc-option,-march=skylake,$(call cc-option,-mtune=skylake)) + cflags-$(CONFIG_MATOM) += $(call cc-option,-march=bonnell) \ + $(call cc-option,-mtune=bonnell,$(call cc-option,-mtune=generic)) cflags-$(CONFIG_GENERIC_CPU) += $(call cc-option,-mtune=generic) KBUILD_CFLAGS += $(cflags-y) diff --git a/arch/x86/Makefile_32.cpu b/arch/x86/Makefile_32.cpu index f3717d3..bf09c78 100644 --- a/arch/x86/Makefile_32.cpu +++ b/arch/x86/Makefile_32.cpu @@ -22,7 +22,18 @@ cflags-$(CONFIG_MK6) += -march=k6 # Please note, that patches that add -march=athlon-xp and friends are pointless. # They make zero difference whatsosever to performance at this time. cflags-$(CONFIG_MK7) += -march=athlon +cflags-$(CONFIG_MNATIVE) += $(call cc-option,-march=native) cflags-$(CONFIG_MK8) += $(call cc-option,-march=k8,-march=athlon) +cflags-$(CONFIG_MK8SSE3) += $(call cc-option,-march=k8-sse3,-march=athlon) +cflags-$(CONFIG_MK10) += $(call cc-option,-march=amdfam10,-march=athlon) +cflags-$(CONFIG_MBARCELONA) += $(call cc-option,-march=barcelona,-march=athlon) +cflags-$(CONFIG_MBOBCAT) += $(call cc-option,-march=btver1,-march=athlon) +cflags-$(CONFIG_MJAGUAR) += $(call cc-option,-march=btver2,-march=athlon) +cflags-$(CONFIG_MBULLDOZER) += $(call cc-option,-march=bdver1,-march=athlon) +cflags-$(CONFIG_MPILEDRIVER) += $(call cc-option,-march=bdver2,-march=athlon) +cflags-$(CONFIG_MSTEAMROLLER) += $(call cc-option,-march=bdver3,-march=athlon) +cflags-$(CONFIG_MEXCAVATOR) += $(call cc-option,-march=bdver4,-march=athlon) +cflags-$(CONFIG_MPCK) += $(call cc-option,-march=znver1,-march=athlon) cflags-$(CONFIG_MCRUSOE) += -march=i686 -falign-functions=0 -falign-jumps=0 -falign-loops=0 cflags-$(CONFIG_MEFFICEON) += -march=i686 $(call tune,pentium3) -falign-functions=0 -falign-jumps=0 -falign-loops=0 cflags-$(CONFIG_MWINCHIPC6) += $(call cc-option,-march=winchip-c6,-march=i586) @@ -31,8 +42,16 @@ cflags-$(CONFIG_MCYRIXIII) += $(call cc-option,-march=c3,-march=i486) -falign-fu cflags-$(CONFIG_MVIAC3_2) += $(call cc-option,-march=c3-2,-march=i686) cflags-$(CONFIG_MVIAC7) += -march=i686 cflags-$(CONFIG_MCORE2) += -march=i686 $(call tune,core2) -cflags-$(CONFIG_MATOM) += $(call cc-option,-march=atom,$(call cc-option,-march=core2,-march=i686)) \ - $(call cc-option,-mtune=atom,$(call cc-option,-mtune=generic)) +cflags-$(CONFIG_MNEHALEM) += -march=i686 $(call tune,nehalem) +cflags-$(CONFIG_MWESTMERE) += -march=i686 $(call tune,westmere) +cflags-$(CONFIG_MSILVERMONT) += -march=i686 $(call tune,silvermont) +cflags-$(CONFIG_MSANDYBRIDGE) += -march=i686 $(call tune,sandybridge) +cflags-$(CONFIG_MIVYBRIDGE) += -march=i686 $(call tune,ivybridge) +cflags-$(CONFIG_MHASWELL) += -march=i686 $(call tune,haswell) +cflags-$(CONFIG_MBROADWELL) += -march=i686 $(call tune,broadwell) +cflags-$(CONFIG_MSKYLAKE) += -march=i686 $(call tune,skylake) +cflags-$(CONFIG_MATOM) += $(call cc-option,-march=bonnell,$(call cc-option,-march=core2,-march=i686)) \ + $(call cc-option,-mtune=bonnell,$(call cc-option,-mtune=generic)) # AMD Elan support cflags-$(CONFIG_MELAN) += -march=i486 diff --git a/arch/x86/include/asm/module.h b/arch/x86/include/asm/module.h index e3b7819..343be66 100644 --- a/arch/x86/include/asm/module.h +++ b/arch/x86/include/asm/module.h @@ -15,6 +15,24 @@ #define MODULE_PROC_FAMILY "586MMX " #elif defined CONFIG_MCORE2 #define MODULE_PROC_FAMILY "CORE2 " +#elif defined CONFIG_MNATIVE +#define MODULE_PROC_FAMILY "NATIVE " +#elif defined CONFIG_MNEHALEM +#define MODULE_PROC_FAMILY "NEHALEM " +#elif defined CONFIG_MWESTMERE +#define MODULE_PROC_FAMILY "WESTMERE " +#elif defined CONFIG_MSILVERMONT +#define MODULE_PROC_FAMILY "SILVERMONT " +#elif defined CONFIG_MSANDYBRIDGE +#define MODULE_PROC_FAMILY "SANDYBRIDGE " +#elif defined CONFIG_MIVYBRIDGE +#define MODULE_PROC_FAMILY "IVYBRIDGE " +#elif defined CONFIG_MHASWELL +#define MODULE_PROC_FAMILY "HASWELL " +#elif defined CONFIG_MBROADWELL +#define MODULE_PROC_FAMILY "BROADWELL " +#elif defined CONFIG_MSKYLAKE +#define MODULE_PROC_FAMILY "SKYLAKE " #elif defined CONFIG_MATOM #define MODULE_PROC_FAMILY "ATOM " #elif defined CONFIG_M686 @@ -33,6 +51,26 @@ #define MODULE_PROC_FAMILY "K7 " #elif defined CONFIG_MK8 #define MODULE_PROC_FAMILY "K8 " +#elif defined CONFIG_MK8SSE3 +#define MODULE_PROC_FAMILY "K8SSE3 " +#elif defined CONFIG_MK10 +#define MODULE_PROC_FAMILY "K10 " +#elif defined CONFIG_MBARCELONA +#define MODULE_PROC_FAMILY "BARCELONA " +#elif defined CONFIG_MBOBCAT +#define MODULE_PROC_FAMILY "BOBCAT " +#elif defined CONFIG_MBULLDOZER +#define MODULE_PROC_FAMILY "BULLDOZER " +#elif defined CONFIG_MPILEDRIVER +#define MODULE_PROC_FAMILY "PILEDRIVER " +#elif defined CONFIG_MSTEAMROLLER +#define MODULE_PROC_FAMILY "STEAMROLLER " +#elif defined CONFIG_MJAGUAR +#define MODULE_PROC_FAMILY "JAGUAR " +#elif defined CONFIG_MEXCAVATOR +#define MODULE_PROC_FAMILY "EXCAVATOR " +#elif defined CONFIG_MPCK +#define MODULE_PROC_FAMILY "PCK " #elif defined CONFIG_MELAN #define MODULE_PROC_FAMILY "ELAN " #elif defined CONFIG_MCRUSOE diff --git a/block/Kconfig.iosched b/block/Kconfig.iosched index fd2cefa..9defb30 100644 --- a/block/Kconfig.iosched +++ b/block/Kconfig.iosched @@ -39,6 +39,26 @@ config CFQ_GROUP_IOSCHED ---help--- Enable group IO scheduling in CFQ. +config IOSCHED_BFQ_SQ + tristate "BFQ-SQ I/O scheduler" + default n + ---help--- + The BFQ-SQ I/O scheduler (for legacy blk: SQ stands for + SingleQueue) distributes bandwidth among all processes + according to their weights, regardless of the device + parameters and with any workload. It also guarantees a low + latency to interactive and soft real-time applications. + Details in Documentation/block/bfq-iosched.txt + +config BFQ_SQ_GROUP_IOSCHED + bool "BFQ-SQ hierarchical scheduling support" + depends on IOSCHED_BFQ_SQ && BLK_CGROUP + default n + ---help--- + + Enable hierarchical scheduling in BFQ-SQ, using the blkio + (cgroups-v1) or io (cgroups-v2) controller. + choice prompt "Default I/O scheduler" @@ -53,6 +73,16 @@ choice config DEFAULT_CFQ bool "CFQ" if IOSCHED_CFQ=y + config DEFAULT_BFQ_SQ + bool "BFQ-SQ" if IOSCHED_BFQ_SQ=y + help + Selects BFQ-SQ as the default I/O scheduler which will be + used by default for all block devices. + The BFQ-SQ I/O scheduler aims at distributing the bandwidth + as desired, independently of the disk parameters and with + any workload. It also tries to guarantee low latency to + interactive and soft real-time applications. + config DEFAULT_NOOP bool "No-op" @@ -62,8 +92,28 @@ config DEFAULT_IOSCHED string default "deadline" if DEFAULT_DEADLINE default "cfq" if DEFAULT_CFQ + default "bfq-sq" if DEFAULT_BFQ_SQ default "noop" if DEFAULT_NOOP +config MQ_IOSCHED_BFQ + tristate "BFQ-MQ I/O Scheduler" + default y + ---help--- + BFQ I/O scheduler for BLK-MQ. BFQ-MQ distributes bandwidth + among all processes according to their weights, regardless of + the device parameters and with any workload. It also + guarantees a low latency to interactive and soft real-time + applications. Details in Documentation/block/bfq-iosched.txt + +config MQ_BFQ_GROUP_IOSCHED + bool "BFQ-MQ hierarchical scheduling support" + depends on MQ_IOSCHED_BFQ && BLK_CGROUP + default n + ---help--- + + Enable hierarchical scheduling in BFQ-MQ, using the blkio + (cgroups-v1) or io (cgroups-v2) controller. + config MQ_IOSCHED_DEADLINE tristate "MQ deadline I/O scheduler" default y diff --git a/block/Makefile b/block/Makefile index 2b281cf..40b9b5d 100644 --- a/block/Makefile +++ b/block/Makefile @@ -23,6 +23,8 @@ obj-$(CONFIG_MQ_IOSCHED_DEADLINE) += mq-deadline.o obj-$(CONFIG_MQ_IOSCHED_KYBER) += kyber-iosched.o bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o obj-$(CONFIG_IOSCHED_BFQ) += bfq.o +obj-$(CONFIG_IOSCHED_BFQ_SQ) += bfq-sq-iosched.o +obj-$(CONFIG_MQ_IOSCHED_BFQ) += bfq-mq-iosched.o obj-$(CONFIG_BLOCK_COMPAT) += compat_ioctl.o obj-$(CONFIG_BLK_CMDLINE_PARSER) += cmdline-parser.o diff --git b/block/bfq-cgroup-included.c b/block/bfq-cgroup-included.c new file mode 100644 index 0000000..631e53d --- /dev/null +++ b/block/bfq-cgroup-included.c @@ -0,0 +1,1329 @@ +/* + * BFQ: CGROUPS support. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2015 Paolo Valente + * + * Copyright (C) 2016 Paolo Valente + * + * Licensed under the GPL-2 as detailed in the accompanying COPYING.BFQ + * file. + */ + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + +/* bfqg stats flags */ +enum bfqg_stats_flags { + BFQG_stats_waiting = 0, + BFQG_stats_idling, + BFQG_stats_empty, +}; + +#define BFQG_FLAG_FNS(name) \ +static void bfqg_stats_mark_##name(struct bfqg_stats *stats) \ +{ \ + stats->flags |= (1 << BFQG_stats_##name); \ +} \ +static void bfqg_stats_clear_##name(struct bfqg_stats *stats) \ +{ \ + stats->flags &= ~(1 << BFQG_stats_##name); \ +} \ +static int bfqg_stats_##name(struct bfqg_stats *stats) \ +{ \ + return (stats->flags & (1 << BFQG_stats_##name)) != 0; \ +} \ + +BFQG_FLAG_FNS(waiting) +BFQG_FLAG_FNS(idling) +BFQG_FLAG_FNS(empty) +#undef BFQG_FLAG_FNS + +#ifdef BFQ_MQ +/* This should be called with the scheduler lock held. */ +#else +/* This should be called with the queue_lock held. */ +#endif +static void bfqg_stats_update_group_wait_time(struct bfqg_stats *stats) +{ + unsigned long long now; + + if (!bfqg_stats_waiting(stats)) + return; + + now = sched_clock(); + if (time_after64(now, stats->start_group_wait_time)) + blkg_stat_add(&stats->group_wait_time, + now - stats->start_group_wait_time); + bfqg_stats_clear_waiting(stats); +} + +#ifdef BFQ_MQ +/* This should be called with the scheduler lock held. */ +#else +/* This should be called with the queue_lock held. */ +#endif +static void bfqg_stats_set_start_group_wait_time(struct bfq_group *bfqg, + struct bfq_group *curr_bfqg) +{ + struct bfqg_stats *stats = &bfqg->stats; + + if (bfqg_stats_waiting(stats)) + return; + if (bfqg == curr_bfqg) + return; + stats->start_group_wait_time = sched_clock(); + bfqg_stats_mark_waiting(stats); +} + +#ifdef BFQ_MQ +/* This should be called with the scheduler lock held. */ +#else +/* This should be called with the queue_lock held. */ +#endif +static void bfqg_stats_end_empty_time(struct bfqg_stats *stats) +{ + unsigned long long now; + + if (!bfqg_stats_empty(stats)) + return; + + now = sched_clock(); + if (time_after64(now, stats->start_empty_time)) + blkg_stat_add(&stats->empty_time, + now - stats->start_empty_time); + bfqg_stats_clear_empty(stats); +} + +static void bfqg_stats_update_dequeue(struct bfq_group *bfqg) +{ + blkg_stat_add(&bfqg->stats.dequeue, 1); +} + +static void bfqg_stats_set_start_empty_time(struct bfq_group *bfqg) +{ + struct bfqg_stats *stats = &bfqg->stats; + + if (blkg_rwstat_total(&stats->queued)) + return; + + /* + * group is already marked empty. This can happen if bfqq got new + * request in parent group and moved to this group while being added + * to service tree. Just ignore the event and move on. + */ + if (bfqg_stats_empty(stats)) + return; + + stats->start_empty_time = sched_clock(); + bfqg_stats_mark_empty(stats); +} + +static void bfqg_stats_update_idle_time(struct bfq_group *bfqg) +{ + struct bfqg_stats *stats = &bfqg->stats; + + if (bfqg_stats_idling(stats)) { + unsigned long long now = sched_clock(); + + if (time_after64(now, stats->start_idle_time)) + blkg_stat_add(&stats->idle_time, + now - stats->start_idle_time); + bfqg_stats_clear_idling(stats); + } +} + +static void bfqg_stats_set_start_idle_time(struct bfq_group *bfqg) +{ + struct bfqg_stats *stats = &bfqg->stats; + + stats->start_idle_time = sched_clock(); + bfqg_stats_mark_idling(stats); +} + +static void bfqg_stats_update_avg_queue_size(struct bfq_group *bfqg) +{ + struct bfqg_stats *stats = &bfqg->stats; + + blkg_stat_add(&stats->avg_queue_size_sum, + blkg_rwstat_total(&stats->queued)); + blkg_stat_add(&stats->avg_queue_size_samples, 1); + bfqg_stats_update_group_wait_time(stats); +} + +static struct blkcg_policy blkcg_policy_bfq; + +/* + * blk-cgroup policy-related handlers + * The following functions help in converting between blk-cgroup + * internal structures and BFQ-specific structures. + */ + +static struct bfq_group *pd_to_bfqg(struct blkg_policy_data *pd) +{ + return pd ? container_of(pd, struct bfq_group, pd) : NULL; +} + +static struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg) +{ + return pd_to_blkg(&bfqg->pd); +} + +static struct bfq_group *blkg_to_bfqg(struct blkcg_gq *blkg) +{ + struct blkg_policy_data *pd = blkg_to_pd(blkg, &blkcg_policy_bfq); + + return pd_to_bfqg(pd); +} + +/* + * bfq_group handlers + * The following functions help in navigating the bfq_group hierarchy + * by allowing to find the parent of a bfq_group or the bfq_group + * associated to a bfq_queue. + */ + +static struct bfq_group *bfqg_parent(struct bfq_group *bfqg) +{ + struct blkcg_gq *pblkg = bfqg_to_blkg(bfqg)->parent; + + return pblkg ? blkg_to_bfqg(pblkg) : NULL; +} + +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq) +{ + struct bfq_entity *group_entity = bfqq->entity.parent; + + return group_entity ? container_of(group_entity, struct bfq_group, + entity) : + bfqq->bfqd->root_group; +} + +/* + * The following two functions handle get and put of a bfq_group by + * wrapping the related blk-cgroup hooks. + */ + +static void bfqg_get(struct bfq_group *bfqg) +{ +#ifdef BFQ_MQ + bfqg->ref++; +#else + blkg_get(bfqg_to_blkg(bfqg)); +#endif +} + +static void bfqg_put(struct bfq_group *bfqg) +{ +#ifdef BFQ_MQ + bfqg->ref--; + + BUG_ON(bfqg->ref < 0); + if (bfqg->ref == 0) + kfree(bfqg); +#else + blkg_put(bfqg_to_blkg(bfqg)); +#endif +} + +#ifdef BFQ_MQ +static void bfqg_and_blkg_get(struct bfq_group *bfqg) +{ + /* see comments in bfq_bic_update_cgroup for why refcounting bfqg */ + bfqg_get(bfqg); + + blkg_get(bfqg_to_blkg(bfqg)); +} + +static void bfqg_and_blkg_put(struct bfq_group *bfqg) +{ + bfqg_put(bfqg); + + blkg_put(bfqg_to_blkg(bfqg)); +} +#endif + +static void bfqg_stats_update_io_add(struct bfq_group *bfqg, + struct bfq_queue *bfqq, + unsigned int op) +{ + blkg_rwstat_add(&bfqg->stats.queued, op, 1); + bfqg_stats_end_empty_time(&bfqg->stats); + if (!(bfqq == ((struct bfq_data *)bfqg->bfqd)->in_service_queue)) + bfqg_stats_set_start_group_wait_time(bfqg, bfqq_group(bfqq)); +} + +static void bfqg_stats_update_io_remove(struct bfq_group *bfqg, unsigned int op) +{ + blkg_rwstat_add(&bfqg->stats.queued, op, -1); +} + +static void bfqg_stats_update_io_merged(struct bfq_group *bfqg, unsigned int op) +{ + blkg_rwstat_add(&bfqg->stats.merged, op, 1); +} + +static void bfqg_stats_update_completion(struct bfq_group *bfqg, + uint64_t start_time, uint64_t io_start_time, + unsigned int op) +{ + struct bfqg_stats *stats = &bfqg->stats; + unsigned long long now = sched_clock(); + + if (time_after64(now, io_start_time)) + blkg_rwstat_add(&stats->service_time, op, + now - io_start_time); + if (time_after64(io_start_time, start_time)) + blkg_rwstat_add(&stats->wait_time, op, + io_start_time - start_time); +} + +/* @stats = 0 */ +static void bfqg_stats_reset(struct bfqg_stats *stats) +{ + /* queued stats shouldn't be cleared */ + blkg_rwstat_reset(&stats->merged); + blkg_rwstat_reset(&stats->service_time); + blkg_rwstat_reset(&stats->wait_time); + blkg_stat_reset(&stats->time); + blkg_stat_reset(&stats->avg_queue_size_sum); + blkg_stat_reset(&stats->avg_queue_size_samples); + blkg_stat_reset(&stats->dequeue); + blkg_stat_reset(&stats->group_wait_time); + blkg_stat_reset(&stats->idle_time); + blkg_stat_reset(&stats->empty_time); +} + +/* @to += @from */ +static void bfqg_stats_add_aux(struct bfqg_stats *to, struct bfqg_stats *from) +{ + if (!to || !from) + return; + + /* queued stats shouldn't be cleared */ + blkg_rwstat_add_aux(&to->merged, &from->merged); + blkg_rwstat_add_aux(&to->service_time, &from->service_time); + blkg_rwstat_add_aux(&to->wait_time, &from->wait_time); + blkg_stat_add_aux(&from->time, &from->time); + blkg_stat_add_aux(&to->avg_queue_size_sum, &from->avg_queue_size_sum); + blkg_stat_add_aux(&to->avg_queue_size_samples, + &from->avg_queue_size_samples); + blkg_stat_add_aux(&to->dequeue, &from->dequeue); + blkg_stat_add_aux(&to->group_wait_time, &from->group_wait_time); + blkg_stat_add_aux(&to->idle_time, &from->idle_time); + blkg_stat_add_aux(&to->empty_time, &from->empty_time); +} + +/* + * Transfer @bfqg's stats to its parent's dead_stats so that the ancestors' + * recursive stats can still account for the amount used by this bfqg after + * it's gone. + */ +static void bfqg_stats_xfer_dead(struct bfq_group *bfqg) +{ + struct bfq_group *parent; + + if (!bfqg) /* root_group */ + return; + + parent = bfqg_parent(bfqg); + + lockdep_assert_held(bfqg_to_blkg(bfqg)->q->queue_lock); + + if (unlikely(!parent)) + return; + + bfqg_stats_add_aux(&parent->stats, &bfqg->stats); + bfqg_stats_reset(&bfqg->stats); +} + +static void bfq_init_entity(struct bfq_entity *entity, + struct bfq_group *bfqg) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + entity->weight = entity->new_weight; + entity->orig_weight = entity->new_weight; + if (bfqq) { + bfqq->ioprio = bfqq->new_ioprio; + bfqq->ioprio_class = bfqq->new_ioprio_class; +#ifdef BFQ_MQ + /* + * Make sure that bfqg and its associated blkg do not + * disappear before entity. + */ + bfqg_and_blkg_get(bfqg); +#else + bfqg_get(bfqg); +#endif + } + entity->parent = bfqg->my_entity; /* NULL for root group */ + entity->sched_data = &bfqg->sched_data; +} + +static void bfqg_stats_exit(struct bfqg_stats *stats) +{ + blkg_rwstat_exit(&stats->merged); + blkg_rwstat_exit(&stats->service_time); + blkg_rwstat_exit(&stats->wait_time); + blkg_rwstat_exit(&stats->queued); + blkg_stat_exit(&stats->time); + blkg_stat_exit(&stats->avg_queue_size_sum); + blkg_stat_exit(&stats->avg_queue_size_samples); + blkg_stat_exit(&stats->dequeue); + blkg_stat_exit(&stats->group_wait_time); + blkg_stat_exit(&stats->idle_time); + blkg_stat_exit(&stats->empty_time); +} + +static int bfqg_stats_init(struct bfqg_stats *stats, gfp_t gfp) +{ + if (blkg_rwstat_init(&stats->merged, gfp) || + blkg_rwstat_init(&stats->service_time, gfp) || + blkg_rwstat_init(&stats->wait_time, gfp) || + blkg_rwstat_init(&stats->queued, gfp) || + blkg_stat_init(&stats->time, gfp) || + blkg_stat_init(&stats->avg_queue_size_sum, gfp) || + blkg_stat_init(&stats->avg_queue_size_samples, gfp) || + blkg_stat_init(&stats->dequeue, gfp) || + blkg_stat_init(&stats->group_wait_time, gfp) || + blkg_stat_init(&stats->idle_time, gfp) || + blkg_stat_init(&stats->empty_time, gfp)) { + bfqg_stats_exit(stats); + return -ENOMEM; + } + + return 0; +} + +static struct bfq_group_data *cpd_to_bfqgd(struct blkcg_policy_data *cpd) +{ + return cpd ? container_of(cpd, struct bfq_group_data, pd) : NULL; +} + +static struct bfq_group_data *blkcg_to_bfqgd(struct blkcg *blkcg) +{ + return cpd_to_bfqgd(blkcg_to_cpd(blkcg, &blkcg_policy_bfq)); +} + +static struct blkcg_policy_data *bfq_cpd_alloc(gfp_t gfp) +{ + struct bfq_group_data *bgd; + + bgd = kzalloc(sizeof(*bgd), gfp); + if (!bgd) + return NULL; + return &bgd->pd; +} + +static void bfq_cpd_init(struct blkcg_policy_data *cpd) +{ + struct bfq_group_data *d = cpd_to_bfqgd(cpd); + + d->weight = cgroup_subsys_on_dfl(io_cgrp_subsys) ? + CGROUP_WEIGHT_DFL : BFQ_WEIGHT_LEGACY_DFL; +} + +static void bfq_cpd_free(struct blkcg_policy_data *cpd) +{ + kfree(cpd_to_bfqgd(cpd)); +} + +static struct blkg_policy_data *bfq_pd_alloc(gfp_t gfp, int node) +{ + struct bfq_group *bfqg; + + bfqg = kzalloc_node(sizeof(*bfqg), gfp, node); + if (!bfqg) + return NULL; + + if (bfqg_stats_init(&bfqg->stats, gfp)) { + kfree(bfqg); + return NULL; + } + +#ifdef BFQ_MQ + /* see comments in bfq_bic_update_cgroup for why refcounting */ + bfqg_get(bfqg); +#endif + return &bfqg->pd; +} + +static void bfq_pd_init(struct blkg_policy_data *pd) +{ + struct blkcg_gq *blkg; + struct bfq_group *bfqg; + struct bfq_data *bfqd; + struct bfq_entity *entity; + struct bfq_group_data *d; + + blkg = pd_to_blkg(pd); + BUG_ON(!blkg); + bfqg = blkg_to_bfqg(blkg); + bfqd = blkg->q->elevator->elevator_data; + BUG_ON(bfqg == bfqd->root_group); + entity = &bfqg->entity; + d = blkcg_to_bfqgd(blkg->blkcg); + + entity->orig_weight = entity->weight = entity->new_weight = d->weight; + entity->my_sched_data = &bfqg->sched_data; + bfqg->my_entity = entity; /* + * the root_group's will be set to NULL + * in bfq_init_queue() + */ + bfqg->bfqd = bfqd; + bfqg->active_entities = 0; + bfqg->rq_pos_tree = RB_ROOT; +} + +static void bfq_pd_free(struct blkg_policy_data *pd) +{ + struct bfq_group *bfqg = pd_to_bfqg(pd); + + bfqg_stats_exit(&bfqg->stats); +#ifdef BFQ_MQ + bfqg_put(bfqg); +#else + kfree(bfqg); +#endif +} + +static void bfq_pd_reset_stats(struct blkg_policy_data *pd) +{ + struct bfq_group *bfqg = pd_to_bfqg(pd); + + bfqg_stats_reset(&bfqg->stats); +} + +static void bfq_group_set_parent(struct bfq_group *bfqg, + struct bfq_group *parent) +{ + struct bfq_entity *entity; + + BUG_ON(!parent); + BUG_ON(!bfqg); + BUG_ON(bfqg == parent); + + entity = &bfqg->entity; + entity->parent = parent->my_entity; + entity->sched_data = &parent->sched_data; +} + +static struct bfq_group *bfq_lookup_bfqg(struct bfq_data *bfqd, + struct blkcg *blkcg) +{ + struct blkcg_gq *blkg; + + blkg = blkg_lookup(blkcg, bfqd->queue); + if (likely(blkg)) + return blkg_to_bfqg(blkg); + return NULL; +} + +static struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, + struct blkcg *blkcg) +{ + struct bfq_group *bfqg, *parent; + struct bfq_entity *entity; + + bfqg = bfq_lookup_bfqg(bfqd, blkcg); + + if (unlikely(!bfqg)) + return NULL; + + /* + * Update chain of bfq_groups as we might be handling a leaf group + * which, along with some of its relatives, has not been hooked yet + * to the private hierarchy of BFQ. + */ + entity = &bfqg->entity; + for_each_entity(entity) { + bfqg = container_of(entity, struct bfq_group, entity); + BUG_ON(!bfqg); + if (bfqg != bfqd->root_group) { + parent = bfqg_parent(bfqg); + if (!parent) + parent = bfqd->root_group; + BUG_ON(!parent); + bfq_group_set_parent(bfqg, parent); + } + } + + return bfqg; +} + +static void bfq_pos_tree_add_move(struct bfq_data *bfqd, + struct bfq_queue *bfqq); + +static void bfq_bfqq_expire(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool compensate, + enum bfqq_expiration reason); + +/** + * bfq_bfqq_move - migrate @bfqq to @bfqg. + * @bfqd: queue descriptor. + * @bfqq: the queue to move. + * @bfqg: the group to move to. + * + * Move @bfqq to @bfqg, deactivating it from its old group and reactivating + * it on the new one. Avoid putting the entity on the old group idle tree. + * +#ifdef BFQ_MQ + * Must be called under the scheduler lock, to make sure that the blkg + * owning @bfqg does not disappear (see comments in + * bfq_bic_update_cgroup on guaranteeing the consistency of blkg + * objects). +#else + * Must be called under the queue lock; the cgroup owning @bfqg must + * not disappear (by now this just means that we are called under + * rcu_read_lock()). +#endif + */ +static void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct bfq_group *bfqg) +{ + struct bfq_entity *entity = &bfqq->entity; + + BUG_ON(!bfq_bfqq_busy(bfqq) && !RB_EMPTY_ROOT(&bfqq->sort_list)); + BUG_ON(!RB_EMPTY_ROOT(&bfqq->sort_list) && !entity->on_st); + BUG_ON(bfq_bfqq_busy(bfqq) && RB_EMPTY_ROOT(&bfqq->sort_list) + && entity->on_st && + bfqq != bfqd->in_service_queue); + BUG_ON(!bfq_bfqq_busy(bfqq) && bfqq == bfqd->in_service_queue); + + /* If bfqq is empty, then bfq_bfqq_expire also invokes + * bfq_del_bfqq_busy, thereby removing bfqq and its entity + * from data structures related to current group. Otherwise we + * need to remove bfqq explicitly with bfq_deactivate_bfqq, as + * we do below. + */ + if (bfqq == bfqd->in_service_queue) + bfq_bfqq_expire(bfqd, bfqd->in_service_queue, + false, BFQ_BFQQ_PREEMPTED); + + BUG_ON(entity->on_st && !bfq_bfqq_busy(bfqq) + && &bfq_entity_service_tree(entity)->idle != + entity->tree); + + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_bfqq_busy(bfqq)); + + if (bfq_bfqq_busy(bfqq)) + bfq_deactivate_bfqq(bfqd, bfqq, false, false); + else if (entity->on_st) { + BUG_ON(&bfq_entity_service_tree(entity)->idle != + entity->tree); + bfq_put_idle_entity(bfq_entity_service_tree(entity), entity); + } +#ifdef BFQ_MQ + bfqg_and_blkg_put(bfqq_group(bfqq)); +#else + bfqg_put(bfqq_group(bfqq)); +#endif + + entity->parent = bfqg->my_entity; + entity->sched_data = &bfqg->sched_data; +#ifdef BFQ_MQ + /* pin down bfqg and its associated blkg */ + bfqg_and_blkg_get(bfqg); +#else + bfqg_get(bfqg); +#endif + + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_bfqq_busy(bfqq)); + if (bfq_bfqq_busy(bfqq)) { + bfq_pos_tree_add_move(bfqd, bfqq); + bfq_activate_bfqq(bfqd, bfqq); + } + + if (!bfqd->in_service_queue && !bfqd->rq_in_driver) + bfq_schedule_dispatch(bfqd); + BUG_ON(entity->on_st && !bfq_bfqq_busy(bfqq) + && &bfq_entity_service_tree(entity)->idle != + entity->tree); +} + +/** + * __bfq_bic_change_cgroup - move @bic to @cgroup. + * @bfqd: the queue descriptor. + * @bic: the bic to move. + * @blkcg: the blk-cgroup to move to. + * +#ifdef BFQ_MQ + * Move bic to blkcg, assuming that bfqd->lock is held; which makes + * sure that the reference to cgroup is valid across the call (see + * comments in bfq_bic_update_cgroup on this issue) +#else + * Move bic to blkcg, assuming that bfqd->queue is locked; the caller + * has to make sure that the reference to cgroup is valid across the call. +#endif + * + * NOTE: an alternative approach might have been to store the current + * cgroup in bfqq and getting a reference to it, reducing the lookup + * time here, at the price of slightly more complex code. + */ +static struct bfq_group *__bfq_bic_change_cgroup(struct bfq_data *bfqd, + struct bfq_io_cq *bic, + struct blkcg *blkcg) +{ + struct bfq_queue *async_bfqq = bic_to_bfqq(bic, 0); + struct bfq_queue *sync_bfqq = bic_to_bfqq(bic, 1); + struct bfq_group *bfqg; + struct bfq_entity *entity; + + bfqg = bfq_find_set_group(bfqd, blkcg); + + if (unlikely(!bfqg)) + bfqg = bfqd->root_group; + + if (async_bfqq) { + entity = &async_bfqq->entity; + + if (entity->sched_data != &bfqg->sched_data) { + bic_set_bfqq(bic, NULL, 0); + bfq_log_bfqq(bfqd, async_bfqq, + "bic_change_group: %p %d", + async_bfqq, + async_bfqq->ref); + bfq_put_queue(async_bfqq); + } + } + + if (sync_bfqq) { + entity = &sync_bfqq->entity; + if (entity->sched_data != &bfqg->sched_data) + bfq_bfqq_move(bfqd, sync_bfqq, bfqg); + } + + return bfqg; +} + +static void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) +{ + struct bfq_data *bfqd = bic_to_bfqd(bic); + struct bfq_group *bfqg = NULL; + uint64_t serial_nr; + + rcu_read_lock(); + serial_nr = bio_blkcg(bio)->css.serial_nr; + + /* + * Check whether blkcg has changed. The condition may trigger + * spuriously on a newly created cic but there's no harm. + */ + if (unlikely(!bfqd) || likely(bic->blkcg_serial_nr == serial_nr)) + goto out; + + bfqg = __bfq_bic_change_cgroup(bfqd, bic, bio_blkcg(bio)); +#ifdef BFQ_MQ + /* + * Update blkg_path for bfq_log_* functions. We cache this + * path, and update it here, for the following + * reasons. Operations on blkg objects in blk-cgroup are + * protected with the request_queue lock, and not with the + * lock that protects the instances of this scheduler + * (bfqd->lock). This exposes BFQ to the following sort of + * race. + * + * The blkg_lookup performed in bfq_get_queue, protected + * through rcu, may happen to return the address of a copy of + * the original blkg. If this is the case, then the + * bfqg_and_blkg_get performed in bfq_get_queue, to pin down + * the blkg, is useless: it does not prevent blk-cgroup code + * from destroying both the original blkg and all objects + * directly or indirectly referred by the copy of the + * blkg. + * + * On the bright side, destroy operations on a blkg invoke, as + * a first step, hooks of the scheduler associated with the + * blkg. And these hooks are executed with bfqd->lock held for + * BFQ. As a consequence, for any blkg associated with the + * request queue this instance of the scheduler is attached + * to, we are guaranteed that such a blkg is not destroyed, and + * that all the pointers it contains are consistent, while we + * are holding bfqd->lock. A blkg_lookup performed with + * bfqd->lock held then returns a fully consistent blkg, which + * remains consistent until this lock is held. + * + * Thanks to the last fact, and to the fact that: (1) bfqg has + * been obtained through a blkg_lookup in the above + * assignment, and (2) bfqd->lock is being held, here we can + * safely use the policy data for the involved blkg (i.e., the + * field bfqg->pd) to get to the blkg associated with bfqg, + * and then we can safely use any field of blkg. After we + * release bfqd->lock, even just getting blkg through this + * bfqg may cause dangling references to be traversed, as + * bfqg->pd may not exist any more. + * + * In view of the above facts, here we cache, in the bfqg, any + * blkg data we may need for this bic, and for its associated + * bfq_queue. As of now, we need to cache only the path of the + * blkg, which is used in the bfq_log_* functions. + * + * Finally, note that bfqg itself needs to be protected from + * destruction on the blkg_free of the original blkg (which + * invokes bfq_pd_free). We use an additional private + * refcounter for bfqg, to let it disappear only after no + * bfq_queue refers to it any longer. + */ + blkg_path(bfqg_to_blkg(bfqg), bfqg->blkg_path, sizeof(bfqg->blkg_path)); +#endif + bic->blkcg_serial_nr = serial_nr; +out: + rcu_read_unlock(); +} + +/** + * bfq_flush_idle_tree - deactivate any entity on the idle tree of @st. + * @st: the service tree being flushed. + */ +static void bfq_flush_idle_tree(struct bfq_service_tree *st) +{ + struct bfq_entity *entity = st->first_idle; + + for (; entity ; entity = st->first_idle) + __bfq_deactivate_entity(entity, false); +} + +/** + * bfq_reparent_leaf_entity - move leaf entity to the root_group. + * @bfqd: the device data structure with the root group. + * @entity: the entity to move. + */ +static void bfq_reparent_leaf_entity(struct bfq_data *bfqd, + struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + BUG_ON(!bfqq); + bfq_bfqq_move(bfqd, bfqq, bfqd->root_group); +} + +/** + * bfq_reparent_active_entities - move to the root group all active + * entities. + * @bfqd: the device data structure with the root group. + * @bfqg: the group to move from. + * @st: the service tree with the entities. + */ +static void bfq_reparent_active_entities(struct bfq_data *bfqd, + struct bfq_group *bfqg, + struct bfq_service_tree *st) +{ + struct rb_root *active = &st->active; + struct bfq_entity *entity = NULL; + + if (!RB_EMPTY_ROOT(&st->active)) + entity = bfq_entity_of(rb_first(active)); + + for (; entity ; entity = bfq_entity_of(rb_first(active))) + bfq_reparent_leaf_entity(bfqd, entity); + + if (bfqg->sched_data.in_service_entity) + bfq_reparent_leaf_entity(bfqd, + bfqg->sched_data.in_service_entity); +} + +/** + * bfq_pd_offline - deactivate the entity associated with @pd, + * and reparent its children entities. + * @pd: descriptor of the policy going offline. + * + * blkio already grabs the queue_lock for us, so no need to use + * RCU-based magic + */ +static void bfq_pd_offline(struct blkg_policy_data *pd) +{ + struct bfq_service_tree *st; + struct bfq_group *bfqg; + struct bfq_data *bfqd; + struct bfq_entity *entity; +#ifdef BFQ_MQ + unsigned long flags; +#endif + int i; + + BUG_ON(!pd); + bfqg = pd_to_bfqg(pd); + BUG_ON(!bfqg); + bfqd = bfqg->bfqd; + BUG_ON(bfqd && !bfqd->root_group); + + entity = bfqg->my_entity; + + if (!entity) /* root group */ + return; + +#ifdef BFQ_MQ + spin_lock_irqsave(&bfqd->lock, flags); +#endif + + /* + * Empty all service_trees belonging to this group before + * deactivating the group itself. + */ + for (i = 0; i < BFQ_IOPRIO_CLASSES; i++) { + BUG_ON(!bfqg->sched_data.service_tree); + st = bfqg->sched_data.service_tree + i; + /* + * The idle tree may still contain bfq_queues belonging + * to exited task because they never migrated to a different + * cgroup from the one being destroyed now. + */ + bfq_flush_idle_tree(st); + + /* + * It may happen that some queues are still active + * (busy) upon group destruction (if the corresponding + * processes have been forced to terminate). We move + * all the leaf entities corresponding to these queues + * to the root_group. + * Also, it may happen that the group has an entity + * in service, which is disconnected from the active + * tree: it must be moved, too. + * There is no need to put the sync queues, as the + * scheduler has taken no reference. + */ + bfq_reparent_active_entities(bfqd, bfqg, st); + BUG_ON(!RB_EMPTY_ROOT(&st->active)); + BUG_ON(!RB_EMPTY_ROOT(&st->idle)); + } + BUG_ON(bfqg->sched_data.next_in_service); + BUG_ON(bfqg->sched_data.in_service_entity); + + __bfq_deactivate_entity(entity, false); + bfq_put_async_queues(bfqd, bfqg); + +#ifdef BFQ_MQ + spin_unlock_irqrestore(&bfqd->lock, flags); +#endif + /* + * @blkg is going offline and will be ignored by + * blkg_[rw]stat_recursive_sum(). Transfer stats to the parent so + * that they don't get lost. If IOs complete after this point, the + * stats for them will be lost. Oh well... + */ + bfqg_stats_xfer_dead(bfqg); +} + +static void bfq_end_wr_async(struct bfq_data *bfqd) +{ + struct blkcg_gq *blkg; + + list_for_each_entry(blkg, &bfqd->queue->blkg_list, q_node) { + struct bfq_group *bfqg = blkg_to_bfqg(blkg); + BUG_ON(!bfqg); + + bfq_end_wr_async_queues(bfqd, bfqg); + } + bfq_end_wr_async_queues(bfqd, bfqd->root_group); +} + +static int bfq_io_show_weight(struct seq_file *sf, void *v) +{ + struct blkcg *blkcg = css_to_blkcg(seq_css(sf)); + struct bfq_group_data *bfqgd = blkcg_to_bfqgd(blkcg); + unsigned int val = 0; + + if (bfqgd) + val = bfqgd->weight; + + seq_printf(sf, "%u\n", val); + + return 0; +} + +static int bfq_io_set_weight_legacy(struct cgroup_subsys_state *css, + struct cftype *cftype, + u64 val) +{ + struct blkcg *blkcg = css_to_blkcg(css); + struct bfq_group_data *bfqgd = blkcg_to_bfqgd(blkcg); + struct blkcg_gq *blkg; + int ret = -ERANGE; + + if (val < BFQ_MIN_WEIGHT || val > BFQ_MAX_WEIGHT) + return ret; + + ret = 0; + spin_lock_irq(&blkcg->lock); + bfqgd->weight = (unsigned short)val; + hlist_for_each_entry(blkg, &blkcg->blkg_list, blkcg_node) { + struct bfq_group *bfqg = blkg_to_bfqg(blkg); + + if (!bfqg) + continue; + /* + * Setting the prio_changed flag of the entity + * to 1 with new_weight == weight would re-set + * the value of the weight to its ioprio mapping. + * Set the flag only if necessary. + */ + if ((unsigned short)val != bfqg->entity.new_weight) { + bfqg->entity.new_weight = (unsigned short)val; + /* + * Make sure that the above new value has been + * stored in bfqg->entity.new_weight before + * setting the prio_changed flag. In fact, + * this flag may be read asynchronously (in + * critical sections protected by a different + * lock than that held here), and finding this + * flag set may cause the execution of the code + * for updating parameters whose value may + * depend also on bfqg->entity.new_weight (in + * __bfq_entity_update_weight_prio). + * This barrier makes sure that the new value + * of bfqg->entity.new_weight is correctly + * seen in that code. + */ + smp_wmb(); + bfqg->entity.prio_changed = 1; + } + } + spin_unlock_irq(&blkcg->lock); + + return ret; +} + +static ssize_t bfq_io_set_weight(struct kernfs_open_file *of, + char *buf, size_t nbytes, + loff_t off) +{ + u64 weight; + /* First unsigned long found in the file is used */ + int ret = kstrtoull(strim(buf), 0, &weight); + + if (ret) + return ret; + + return bfq_io_set_weight_legacy(of_css(of), NULL, weight); +} + +static int bfqg_print_stat(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), blkg_prfill_stat, + &blkcg_policy_bfq, seq_cft(sf)->private, false); + return 0; +} + +static int bfqg_print_rwstat(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), blkg_prfill_rwstat, + &blkcg_policy_bfq, seq_cft(sf)->private, true); + return 0; +} + +static u64 bfqg_prfill_stat_recursive(struct seq_file *sf, + struct blkg_policy_data *pd, int off) +{ + u64 sum = blkg_stat_recursive_sum(pd_to_blkg(pd), + &blkcg_policy_bfq, off); + return __blkg_prfill_u64(sf, pd, sum); +} + +static u64 bfqg_prfill_rwstat_recursive(struct seq_file *sf, + struct blkg_policy_data *pd, int off) +{ + struct blkg_rwstat sum = blkg_rwstat_recursive_sum(pd_to_blkg(pd), + &blkcg_policy_bfq, + off); + return __blkg_prfill_rwstat(sf, pd, &sum); +} + +static int bfqg_print_stat_recursive(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), + bfqg_prfill_stat_recursive, &blkcg_policy_bfq, + seq_cft(sf)->private, false); + return 0; +} + +static int bfqg_print_rwstat_recursive(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), + bfqg_prfill_rwstat_recursive, &blkcg_policy_bfq, + seq_cft(sf)->private, true); + return 0; +} + +static u64 bfqg_prfill_sectors(struct seq_file *sf, struct blkg_policy_data *pd, + int off) +{ + u64 sum = blkg_rwstat_total(&pd->blkg->stat_bytes); + + return __blkg_prfill_u64(sf, pd, sum >> 9); +} + +static int bfqg_print_stat_sectors(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), + bfqg_prfill_sectors, &blkcg_policy_bfq, 0, false); + return 0; +} + +static u64 bfqg_prfill_sectors_recursive(struct seq_file *sf, + struct blkg_policy_data *pd, int off) +{ + struct blkg_rwstat tmp = blkg_rwstat_recursive_sum(pd->blkg, NULL, + offsetof(struct blkcg_gq, stat_bytes)); + u64 sum = atomic64_read(&tmp.aux_cnt[BLKG_RWSTAT_READ]) + + atomic64_read(&tmp.aux_cnt[BLKG_RWSTAT_WRITE]); + + return __blkg_prfill_u64(sf, pd, sum >> 9); +} + +static int bfqg_print_stat_sectors_recursive(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), + bfqg_prfill_sectors_recursive, &blkcg_policy_bfq, 0, + false); + return 0; +} + + +static u64 bfqg_prfill_avg_queue_size(struct seq_file *sf, + struct blkg_policy_data *pd, int off) +{ + struct bfq_group *bfqg = pd_to_bfqg(pd); + u64 samples = blkg_stat_read(&bfqg->stats.avg_queue_size_samples); + u64 v = 0; + + if (samples) { + v = blkg_stat_read(&bfqg->stats.avg_queue_size_sum); + v = div64_u64(v, samples); + } + __blkg_prfill_u64(sf, pd, v); + return 0; +} + +/* print avg_queue_size */ +static int bfqg_print_avg_queue_size(struct seq_file *sf, void *v) +{ + blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), + bfqg_prfill_avg_queue_size, &blkcg_policy_bfq, + 0, false); + return 0; +} + +static struct bfq_group * +bfq_create_group_hierarchy(struct bfq_data *bfqd, int node) +{ + int ret; + + ret = blkcg_activate_policy(bfqd->queue, &blkcg_policy_bfq); + if (ret) + return NULL; + + return blkg_to_bfqg(bfqd->queue->root_blkg); +} + +#ifdef BFQ_MQ +#define BFQ_CGROUP_FNAME(param) "bfq-mq."#param +#else +#define BFQ_CGROUP_FNAME(param) "bfq-sq."#param +#endif + +static struct cftype bfq_blkcg_legacy_files[] = { + { + .name = BFQ_CGROUP_FNAME(weight), + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = bfq_io_show_weight, + .write_u64 = bfq_io_set_weight_legacy, + }, + + /* statistics, covers only the tasks in the bfqg */ + { + .name = BFQ_CGROUP_FNAME(time), + .private = offsetof(struct bfq_group, stats.time), + .seq_show = bfqg_print_stat, + }, + { + .name = BFQ_CGROUP_FNAME(sectors), + .seq_show = bfqg_print_stat_sectors, + }, + { + .name = BFQ_CGROUP_FNAME(io_service_bytes), + .private = (unsigned long)&blkcg_policy_bfq, + .seq_show = blkg_print_stat_bytes, + }, + { + .name = BFQ_CGROUP_FNAME(io_serviced), + .private = (unsigned long)&blkcg_policy_bfq, + .seq_show = blkg_print_stat_ios, + }, + { + .name = BFQ_CGROUP_FNAME(io_service_time), + .private = offsetof(struct bfq_group, stats.service_time), + .seq_show = bfqg_print_rwstat, + }, + { + .name = BFQ_CGROUP_FNAME(io_wait_time), + .private = offsetof(struct bfq_group, stats.wait_time), + .seq_show = bfqg_print_rwstat, + }, + { + .name = BFQ_CGROUP_FNAME(io_merged), + .private = offsetof(struct bfq_group, stats.merged), + .seq_show = bfqg_print_rwstat, + }, + { + .name = BFQ_CGROUP_FNAME(io_queued), + .private = offsetof(struct bfq_group, stats.queued), + .seq_show = bfqg_print_rwstat, + }, + + /* the same statictics which cover the bfqg and its descendants */ + { + .name = BFQ_CGROUP_FNAME(time_recursive), + .private = offsetof(struct bfq_group, stats.time), + .seq_show = bfqg_print_stat_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(sectors_recursive), + .seq_show = bfqg_print_stat_sectors_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(io_service_bytes_recursive), + .private = (unsigned long)&blkcg_policy_bfq, + .seq_show = blkg_print_stat_bytes_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(io_serviced_recursive), + .private = (unsigned long)&blkcg_policy_bfq, + .seq_show = blkg_print_stat_ios_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(io_service_time_recursive), + .private = offsetof(struct bfq_group, stats.service_time), + .seq_show = bfqg_print_rwstat_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(io_wait_time_recursive), + .private = offsetof(struct bfq_group, stats.wait_time), + .seq_show = bfqg_print_rwstat_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(io_merged_recursive), + .private = offsetof(struct bfq_group, stats.merged), + .seq_show = bfqg_print_rwstat_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(io_queued_recursive), + .private = offsetof(struct bfq_group, stats.queued), + .seq_show = bfqg_print_rwstat_recursive, + }, + { + .name = BFQ_CGROUP_FNAME(avg_queue_size), + .seq_show = bfqg_print_avg_queue_size, + }, + { + .name = BFQ_CGROUP_FNAME(group_wait_time), + .private = offsetof(struct bfq_group, stats.group_wait_time), + .seq_show = bfqg_print_stat, + }, + { + .name = BFQ_CGROUP_FNAME(idle_time), + .private = offsetof(struct bfq_group, stats.idle_time), + .seq_show = bfqg_print_stat, + }, + { + .name = BFQ_CGROUP_FNAME(empty_time), + .private = offsetof(struct bfq_group, stats.empty_time), + .seq_show = bfqg_print_stat, + }, + { + .name = BFQ_CGROUP_FNAME(dequeue), + .private = offsetof(struct bfq_group, stats.dequeue), + .seq_show = bfqg_print_stat, + }, + { } /* terminate */ +}; + +static struct cftype bfq_blkg_files[] = { + { + .name = BFQ_CGROUP_FNAME(weight), + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = bfq_io_show_weight, + .write = bfq_io_set_weight, + }, + {} /* terminate */ +}; + +#undef BFQ_CGROUP_FNAME + +#else /* BFQ_GROUP_IOSCHED_ENABLED */ + +static inline void bfqg_stats_update_io_add(struct bfq_group *bfqg, + struct bfq_queue *bfqq, unsigned int op) { } +static inline void +bfqg_stats_update_io_remove(struct bfq_group *bfqg, unsigned int op) { } +static inline void +bfqg_stats_update_io_merged(struct bfq_group *bfqg, unsigned int op) { } +static inline void bfqg_stats_update_completion(struct bfq_group *bfqg, + uint64_t start_time, uint64_t io_start_time, + unsigned int op) { } +static inline void +bfqg_stats_set_start_group_wait_time(struct bfq_group *bfqg, + struct bfq_group *curr_bfqg) { } +static inline void bfqg_stats_end_empty_time(struct bfqg_stats *stats) { } +static inline void bfqg_stats_update_dequeue(struct bfq_group *bfqg) { } +static inline void bfqg_stats_set_start_empty_time(struct bfq_group *bfqg) { } +static inline void bfqg_stats_update_idle_time(struct bfq_group *bfqg) { } +static inline void bfqg_stats_set_start_idle_time(struct bfq_group *bfqg) { } +static inline void bfqg_stats_update_avg_queue_size(struct bfq_group *bfqg) { } + +static void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct bfq_group *bfqg) {} + +static void bfq_init_entity(struct bfq_entity *entity, + struct bfq_group *bfqg) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + entity->weight = entity->new_weight; + entity->orig_weight = entity->new_weight; + if (bfqq) { + bfqq->ioprio = bfqq->new_ioprio; + bfqq->ioprio_class = bfqq->new_ioprio_class; + } + entity->sched_data = &bfqg->sched_data; +} + +static void bfq_bic_update_cgroup(struct bfq_io_cq *bic, struct bio *bio) {} + +static void bfq_end_wr_async(struct bfq_data *bfqd) +{ + bfq_end_wr_async_queues(bfqd, bfqd->root_group); +} + +static struct bfq_group *bfq_find_set_group(struct bfq_data *bfqd, + struct blkcg *blkcg) +{ + return bfqd->root_group; +} + +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq) +{ + return bfqq->bfqd->root_group; +} + +static struct bfq_group * +bfq_create_group_hierarchy(struct bfq_data *bfqd, int node) +{ + struct bfq_group *bfqg; + int i; + + bfqg = kmalloc_node(sizeof(*bfqg), GFP_KERNEL | __GFP_ZERO, node); + if (!bfqg) + return NULL; + + for (i = 0; i < BFQ_IOPRIO_CLASSES; i++) + bfqg->sched_data.service_tree[i] = BFQ_SERVICE_TREE_INIT; + + return bfqg; +} +#endif diff --git b/block/bfq-ioc.c b/block/bfq-ioc.c new file mode 100644 index 0000000..fb7bb8f --- /dev/null +++ b/block/bfq-ioc.c @@ -0,0 +1,36 @@ +/* + * BFQ: I/O context handling. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2010 Paolo Valente + */ + +/** + * icq_to_bic - convert iocontext queue structure to bfq_io_cq. + * @icq: the iocontext queue. + */ +static struct bfq_io_cq *icq_to_bic(struct io_cq *icq) +{ + /* bic->icq is the first member, %NULL will convert to %NULL */ + return container_of(icq, struct bfq_io_cq, icq); +} + +/** + * bfq_bic_lookup - search into @ioc a bic associated to @bfqd. + * @bfqd: the lookup key. + * @ioc: the io_context of the process doing I/O. + * + * Queue lock must be held. + */ +static struct bfq_io_cq *bfq_bic_lookup(struct bfq_data *bfqd, + struct io_context *ioc) +{ + if (ioc) + return icq_to_bic(ioc_lookup_icq(ioc, bfqd->queue)); + return NULL; +} diff --git b/block/bfq-mq-iosched.c b/block/bfq-mq-iosched.c new file mode 100644 index 0000000..8c5dce7 --- /dev/null +++ b/block/bfq-mq-iosched.c @@ -0,0 +1,5693 @@ +/* + * Budget Fair Queueing (BFQ) I/O scheduler. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2015 Paolo Valente + * + * Copyright (C) 2017 Paolo Valente + * + * Licensed under the GPL-2 as detailed in the accompanying COPYING.BFQ + * file. + * + * BFQ is a proportional-share I/O scheduler, with some extra + * low-latency capabilities. BFQ also supports full hierarchical + * scheduling through cgroups. Next paragraphs provide an introduction + * on BFQ inner workings. Details on BFQ benefits and usage can be + * found in Documentation/block/bfq-iosched.txt. + * + * BFQ is a proportional-share storage-I/O scheduling algorithm based + * on the slice-by-slice service scheme of CFQ. But BFQ assigns + * budgets, measured in number of sectors, to processes instead of + * time slices. The device is not granted to the in-service process + * for a given time slice, but until it has exhausted its assigned + * budget. This change from the time to the service domain enables BFQ + * to distribute the device throughput among processes as desired, + * without any distortion due to throughput fluctuations, or to device + * internal queueing. BFQ uses an ad hoc internal scheduler, called + * B-WF2Q+, to schedule processes according to their budgets. More + * precisely, BFQ schedules queues associated with processes. Thanks to + * the accurate policy of B-WF2Q+, BFQ can afford to assign high + * budgets to I/O-bound processes issuing sequential requests (to + * boost the throughput), and yet guarantee a low latency to + * interactive and soft real-time applications. + * + * NOTE: if the main or only goal, with a given device, is to achieve + * the maximum-possible throughput at all times, then do switch off + * all low-latency heuristics for that device, by setting low_latency + * to 0. + * + * BFQ is described in [1], where also a reference to the initial, more + * theoretical paper on BFQ can be found. The interested reader can find + * in the latter paper full details on the main algorithm, as well as + * formulas of the guarantees and formal proofs of all the properties. + * With respect to the version of BFQ presented in these papers, this + * implementation adds a few more heuristics, such as the one that + * guarantees a low latency to soft real-time applications, and a + * hierarchical extension based on H-WF2Q+. + * + * B-WF2Q+ is based on WF2Q+, that is described in [2], together with + * H-WF2Q+, while the augmented tree used to implement B-WF2Q+ with O(log N) + * complexity derives from the one introduced with EEVDF in [3]. + * + * [1] P. Valente, A. Avanzini, "Evolution of the BFQ Storage I/O + * Scheduler", Proceedings of the First Workshop on Mobile System + * Technologies (MST-2015), May 2015. + * http://algogroup.unimore.it/people/paolo/disk_sched/mst-2015.pdf + * + * http://algogroup.unimo.it/people/paolo/disk_sched/bf1-v1-suite-results.pdf + * + * [2] Jon C.R. Bennett and H. Zhang, ``Hierarchical Packet Fair Queueing + * Algorithms,'' IEEE/ACM Transactions on Networking, 5(5):675-689, + * Oct 1997. + * + * http://www.cs.cmu.edu/~hzhang/papers/TON-97-Oct.ps.gz + * + * [3] I. Stoica and H. Abdel-Wahab, ``Earliest Eligible Virtual Deadline + * First: A Flexible and Accurate Mechanism for Proportional Share + * Resource Allocation,'' technical report. + * + * http://www.cs.berkeley.edu/~istoica/papers/eevdf-tr-95.pdf + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "blk.h" +#include "blk-mq.h" +#include "blk-mq-tag.h" +#include "blk-mq-sched.h" +#include "bfq-mq.h" + +/* Expiration time of sync (0) and async (1) requests, in ns. */ +static const u64 bfq_fifo_expire[2] = { NSEC_PER_SEC / 4, NSEC_PER_SEC / 8 }; + +/* Maximum backwards seek, in KiB. */ +static const int bfq_back_max = (16 * 1024); + +/* Penalty of a backwards seek, in number of sectors. */ +static const int bfq_back_penalty = 2; + +/* Idling period duration, in ns. */ +static u32 bfq_slice_idle = (NSEC_PER_SEC / 125); + +/* Minimum number of assigned budgets for which stats are safe to compute. */ +static const int bfq_stats_min_budgets = 194; + +/* Default maximum budget values, in sectors and number of requests. */ +static const int bfq_default_max_budget = (16 * 1024); + +/* + * Async to sync throughput distribution is controlled as follows: + * when an async request is served, the entity is charged the number + * of sectors of the request, multiplied by the factor below + */ +static const int bfq_async_charge_factor = 10; + +/* Default timeout values, in jiffies, approximating CFQ defaults. */ +static const int bfq_timeout = (HZ / 8); + +static struct kmem_cache *bfq_pool; + +/* Below this threshold (in ns), we consider thinktime immediate. */ +#define BFQ_MIN_TT (2 * NSEC_PER_MSEC) + +/* hw_tag detection: parallel requests threshold and min samples needed. */ +#define BFQ_HW_QUEUE_THRESHOLD 4 +#define BFQ_HW_QUEUE_SAMPLES 32 + +#define BFQQ_SEEK_THR (sector_t)(8 * 100) +#define BFQQ_SECT_THR_NONROT (sector_t)(2 * 32) +#define BFQQ_CLOSE_THR (sector_t)(8 * 1024) +#define BFQQ_SEEKY(bfqq) (hweight32(bfqq->seek_history) > 32/8) + +/* Min number of samples required to perform peak-rate update */ +#define BFQ_RATE_MIN_SAMPLES 32 +/* Min observation time interval required to perform a peak-rate update (ns) */ +#define BFQ_RATE_MIN_INTERVAL (300*NSEC_PER_MSEC) +/* Target observation time interval for a peak-rate update (ns) */ +#define BFQ_RATE_REF_INTERVAL NSEC_PER_SEC + +/* Shift used for peak rate fixed precision calculations. */ +#define BFQ_RATE_SHIFT 16 + +/* + * By default, BFQ computes the duration of the weight raising for + * interactive applications automatically, using the following formula: + * duration = (R / r) * T, where r is the peak rate of the device, and + * R and T are two reference parameters. + * In particular, R is the peak rate of the reference device (see below), + * and T is a reference time: given the systems that are likely to be + * installed on the reference device according to its speed class, T is + * about the maximum time needed, under BFQ and while reading two files in + * parallel, to load typical large applications on these systems. + * In practice, the slower/faster the device at hand is, the more/less it + * takes to load applications with respect to the reference device. + * Accordingly, the longer/shorter BFQ grants weight raising to interactive + * applications. + * + * BFQ uses four different reference pairs (R, T), depending on: + * . whether the device is rotational or non-rotational; + * . whether the device is slow, such as old or portable HDDs, as well as + * SD cards, or fast, such as newer HDDs and SSDs. + * + * The device's speed class is dynamically (re)detected in + * bfq_update_peak_rate() every time the estimated peak rate is updated. + * + * In the following definitions, R_slow[0]/R_fast[0] and + * T_slow[0]/T_fast[0] are the reference values for a slow/fast + * rotational device, whereas R_slow[1]/R_fast[1] and + * T_slow[1]/T_fast[1] are the reference values for a slow/fast + * non-rotational device. Finally, device_speed_thresh are the + * thresholds used to switch between speed classes. The reference + * rates are not the actual peak rates of the devices used as a + * reference, but slightly lower values. The reason for using these + * slightly lower values is that the peak-rate estimator tends to + * yield slightly lower values than the actual peak rate (it can yield + * the actual peak rate only if there is only one process doing I/O, + * and the process does sequential I/O). + * + * Both the reference peak rates and the thresholds are measured in + * sectors/usec, left-shifted by BFQ_RATE_SHIFT. + */ +static int R_slow[2] = {1000, 10700}; +static int R_fast[2] = {14000, 33000}; +/* + * To improve readability, a conversion function is used to initialize the + * following arrays, which entails that they can be initialized only in a + * function. + */ +static int T_slow[2]; +static int T_fast[2]; +static int device_speed_thresh[2]; + +#define BFQ_SERVICE_TREE_INIT ((struct bfq_service_tree) \ + { RB_ROOT, RB_ROOT, NULL, NULL, 0, 0 }) + +#define RQ_BIC(rq) ((struct bfq_io_cq *) (rq)->elv.priv[0]) +#define RQ_BFQQ(rq) ((rq)->elv.priv[1]) + +/** + * icq_to_bic - convert iocontext queue structure to bfq_io_cq. + * @icq: the iocontext queue. + */ +static struct bfq_io_cq *icq_to_bic(struct io_cq *icq) +{ + /* bic->icq is the first member, %NULL will convert to %NULL */ + return container_of(icq, struct bfq_io_cq, icq); +} + +/** + * bfq_bic_lookup - search into @ioc a bic associated to @bfqd. + * @bfqd: the lookup key. + * @ioc: the io_context of the process doing I/O. + * @q: the request queue. + */ +static struct bfq_io_cq *bfq_bic_lookup(struct bfq_data *bfqd, + struct io_context *ioc, + struct request_queue *q) +{ + if (ioc) { + unsigned long flags; + struct bfq_io_cq *icq; + + spin_lock_irqsave(q->queue_lock, flags); + icq = icq_to_bic(ioc_lookup_icq(ioc, q)); + spin_unlock_irqrestore(q->queue_lock, flags); + + return icq; + } + + return NULL; +} + +/* + * Scheduler run of queue, if there are requests pending and no one in the + * driver that will restart queueing. + */ +static void bfq_schedule_dispatch(struct bfq_data *bfqd) +{ + if (bfqd->queued != 0) { + bfq_log(bfqd, "schedule dispatch"); + blk_mq_run_hw_queues(bfqd->queue, true); + } +} + +#define BFQ_MQ +#include "bfq-sched.c" +#include "bfq-cgroup-included.c" + +#define bfq_class_idle(bfqq) ((bfqq)->ioprio_class == IOPRIO_CLASS_IDLE) +#define bfq_class_rt(bfqq) ((bfqq)->ioprio_class == IOPRIO_CLASS_RT) + +#define bfq_sample_valid(samples) ((samples) > 80) + +/* + * Lifted from AS - choose which of rq1 and rq2 that is best served now. + * We choose the request that is closesr to the head right now. Distance + * behind the head is penalized and only allowed to a certain extent. + */ +static struct request *bfq_choose_req(struct bfq_data *bfqd, + struct request *rq1, + struct request *rq2, + sector_t last) +{ + sector_t s1, s2, d1 = 0, d2 = 0; + unsigned long back_max; +#define BFQ_RQ1_WRAP 0x01 /* request 1 wraps */ +#define BFQ_RQ2_WRAP 0x02 /* request 2 wraps */ + unsigned int wrap = 0; /* bit mask: requests behind the disk head? */ + + if (!rq1 || rq1 == rq2) + return rq2; + if (!rq2) + return rq1; + + if (rq_is_sync(rq1) && !rq_is_sync(rq2)) + return rq1; + else if (rq_is_sync(rq2) && !rq_is_sync(rq1)) + return rq2; + if ((rq1->cmd_flags & REQ_META) && !(rq2->cmd_flags & REQ_META)) + return rq1; + else if ((rq2->cmd_flags & REQ_META) && !(rq1->cmd_flags & REQ_META)) + return rq2; + + s1 = blk_rq_pos(rq1); + s2 = blk_rq_pos(rq2); + + /* + * By definition, 1KiB is 2 sectors. + */ + back_max = bfqd->bfq_back_max * 2; + + /* + * Strict one way elevator _except_ in the case where we allow + * short backward seeks which are biased as twice the cost of a + * similar forward seek. + */ + if (s1 >= last) + d1 = s1 - last; + else if (s1 + back_max >= last) + d1 = (last - s1) * bfqd->bfq_back_penalty; + else + wrap |= BFQ_RQ1_WRAP; + + if (s2 >= last) + d2 = s2 - last; + else if (s2 + back_max >= last) + d2 = (last - s2) * bfqd->bfq_back_penalty; + else + wrap |= BFQ_RQ2_WRAP; + + /* Found required data */ + + /* + * By doing switch() on the bit mask "wrap" we avoid having to + * check two variables for all permutations: --> faster! + */ + switch (wrap) { + case 0: /* common case for CFQ: rq1 and rq2 not wrapped */ + if (d1 < d2) + return rq1; + else if (d2 < d1) + return rq2; + + if (s1 >= s2) + return rq1; + else + return rq2; + + case BFQ_RQ2_WRAP: + return rq1; + case BFQ_RQ1_WRAP: + return rq2; + case (BFQ_RQ1_WRAP|BFQ_RQ2_WRAP): /* both rqs wrapped */ + default: + /* + * Since both rqs are wrapped, + * start with the one that's further behind head + * (--> only *one* back seek required), + * since back seek takes more time than forward. + */ + if (s1 <= s2) + return rq1; + else + return rq2; + } +} + +static struct bfq_queue * +bfq_rq_pos_tree_lookup(struct bfq_data *bfqd, struct rb_root *root, + sector_t sector, struct rb_node **ret_parent, + struct rb_node ***rb_link) +{ + struct rb_node **p, *parent; + struct bfq_queue *bfqq = NULL; + + parent = NULL; + p = &root->rb_node; + while (*p) { + struct rb_node **n; + + parent = *p; + bfqq = rb_entry(parent, struct bfq_queue, pos_node); + + /* + * Sort strictly based on sector. Smallest to the left, + * largest to the right. + */ + if (sector > blk_rq_pos(bfqq->next_rq)) + n = &(*p)->rb_right; + else if (sector < blk_rq_pos(bfqq->next_rq)) + n = &(*p)->rb_left; + else + break; + p = n; + bfqq = NULL; + } + + *ret_parent = parent; + if (rb_link) + *rb_link = p; + + bfq_log(bfqd, "rq_pos_tree_lookup %llu: returning %d", + (unsigned long long) sector, + bfqq ? bfqq->pid : 0); + + return bfqq; +} + +static void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct rb_node **p, *parent; + struct bfq_queue *__bfqq; + + if (bfqq->pos_root) { + rb_erase(&bfqq->pos_node, bfqq->pos_root); + bfqq->pos_root = NULL; + } + + if (bfq_class_idle(bfqq)) + return; + if (!bfqq->next_rq) + return; + + bfqq->pos_root = &bfq_bfqq_to_bfqg(bfqq)->rq_pos_tree; + __bfqq = bfq_rq_pos_tree_lookup(bfqd, bfqq->pos_root, + blk_rq_pos(bfqq->next_rq), &parent, &p); + if (!__bfqq) { + rb_link_node(&bfqq->pos_node, parent, p); + rb_insert_color(&bfqq->pos_node, bfqq->pos_root); + } else + bfqq->pos_root = NULL; +} + +/* + * Tell whether there are active queues or groups with differentiated weights. + */ +static bool bfq_differentiated_weights(struct bfq_data *bfqd) +{ + /* + * For weights to differ, at least one of the trees must contain + * at least two nodes. + */ + return (!RB_EMPTY_ROOT(&bfqd->queue_weights_tree) && + (bfqd->queue_weights_tree.rb_node->rb_left || + bfqd->queue_weights_tree.rb_node->rb_right) +#ifdef BFQ_GROUP_IOSCHED_ENABLED + ) || + (!RB_EMPTY_ROOT(&bfqd->group_weights_tree) && + (bfqd->group_weights_tree.rb_node->rb_left || + bfqd->group_weights_tree.rb_node->rb_right) +#endif + ); +} + +/* + * The following function returns true if every queue must receive the + * same share of the throughput (this condition is used when deciding + * whether idling may be disabled, see the comments in the function + * bfq_bfqq_may_idle()). + * + * Such a scenario occurs when: + * 1) all active queues have the same weight, + * 2) all active groups at the same level in the groups tree have the same + * weight, + * 3) all active groups at the same level in the groups tree have the same + * number of children. + * + * Unfortunately, keeping the necessary state for evaluating exactly the + * above symmetry conditions would be quite complex and time-consuming. + * Therefore this function evaluates, instead, the following stronger + * sub-conditions, for which it is much easier to maintain the needed + * state: + * 1) all active queues have the same weight, + * 2) all active groups have the same weight, + * 3) all active groups have at most one active child each. + * In particular, the last two conditions are always true if hierarchical + * support and the cgroups interface are not enabled, thus no state needs + * to be maintained in this case. + */ +static bool bfq_symmetric_scenario(struct bfq_data *bfqd) +{ + return !bfq_differentiated_weights(bfqd); +} + +/* + * If the weight-counter tree passed as input contains no counter for + * the weight of the input entity, then add that counter; otherwise just + * increment the existing counter. + * + * Note that weight-counter trees contain few nodes in mostly symmetric + * scenarios. For example, if all queues have the same weight, then the + * weight-counter tree for the queues may contain at most one node. + * This holds even if low_latency is on, because weight-raised queues + * are not inserted in the tree. + * In most scenarios, the rate at which nodes are created/destroyed + * should be low too. + */ +static void bfq_weights_tree_add(struct bfq_data *bfqd, + struct bfq_entity *entity, + struct rb_root *root) +{ + struct rb_node **new = &(root->rb_node), *parent = NULL; + + /* + * Do not insert if the entity is already associated with a + * counter, which happens if: + * 1) the entity is associated with a queue, + * 2) a request arrival has caused the queue to become both + * non-weight-raised, and hence change its weight, and + * backlogged; in this respect, each of the two events + * causes an invocation of this function, + * 3) this is the invocation of this function caused by the + * second event. This second invocation is actually useless, + * and we handle this fact by exiting immediately. More + * efficient or clearer solutions might possibly be adopted. + */ + if (entity->weight_counter) + return; + + while (*new) { + struct bfq_weight_counter *__counter = container_of(*new, + struct bfq_weight_counter, + weights_node); + parent = *new; + + if (entity->weight == __counter->weight) { + entity->weight_counter = __counter; + goto inc_counter; + } + if (entity->weight < __counter->weight) + new = &((*new)->rb_left); + else + new = &((*new)->rb_right); + } + + entity->weight_counter = kzalloc(sizeof(struct bfq_weight_counter), + GFP_ATOMIC); + + /* + * In the unlucky event of an allocation failure, we just + * exit. This will cause the weight of entity to not be + * considered in bfq_differentiated_weights, which, in its + * turn, causes the scenario to be deemed wrongly symmetric in + * case entity's weight would have been the only weight making + * the scenario asymmetric. On the bright side, no unbalance + * will however occur when entity becomes inactive again (the + * invocation of this function is triggered by an activation + * of entity). In fact, bfq_weights_tree_remove does nothing + * if !entity->weight_counter. + */ + if (unlikely(!entity->weight_counter)) + return; + + entity->weight_counter->weight = entity->weight; + rb_link_node(&entity->weight_counter->weights_node, parent, new); + rb_insert_color(&entity->weight_counter->weights_node, root); + +inc_counter: + entity->weight_counter->num_active++; +} + +/* + * Decrement the weight counter associated with the entity, and, if the + * counter reaches 0, remove the counter from the tree. + * See the comments to the function bfq_weights_tree_add() for considerations + * about overhead. + */ +static void bfq_weights_tree_remove(struct bfq_data *bfqd, + struct bfq_entity *entity, + struct rb_root *root) +{ + if (!entity->weight_counter) + return; + + BUG_ON(RB_EMPTY_ROOT(root)); + BUG_ON(entity->weight_counter->weight != entity->weight); + + BUG_ON(!entity->weight_counter->num_active); + entity->weight_counter->num_active--; + if (entity->weight_counter->num_active > 0) + goto reset_entity_pointer; + + rb_erase(&entity->weight_counter->weights_node, root); + kfree(entity->weight_counter); + +reset_entity_pointer: + entity->weight_counter = NULL; +} + +/* + * Return expired entry, or NULL to just start from scratch in rbtree. + */ +static struct request *bfq_check_fifo(struct bfq_queue *bfqq, + struct request *last) +{ + struct request *rq; + + if (bfq_bfqq_fifo_expire(bfqq)) + return NULL; + + bfq_mark_bfqq_fifo_expire(bfqq); + + rq = rq_entry_fifo(bfqq->fifo.next); + + if (rq == last || ktime_get_ns() < rq->fifo_time) + return NULL; + + bfq_log_bfqq(bfqq->bfqd, bfqq, "check_fifo: returned %p", rq); + BUG_ON(RB_EMPTY_NODE(&rq->rb_node)); + return rq; +} + +static struct request *bfq_find_next_rq(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + struct request *last) +{ + struct rb_node *rbnext = rb_next(&last->rb_node); + struct rb_node *rbprev = rb_prev(&last->rb_node); + struct request *next, *prev = NULL; + + BUG_ON(list_empty(&bfqq->fifo)); + + /* Follow expired path, else get first next available. */ + next = bfq_check_fifo(bfqq, last); + if (next) { + BUG_ON(next == last); + return next; + } + + BUG_ON(RB_EMPTY_NODE(&last->rb_node)); + + if (rbprev) + prev = rb_entry_rq(rbprev); + + if (rbnext) + next = rb_entry_rq(rbnext); + else { + rbnext = rb_first(&bfqq->sort_list); + if (rbnext && rbnext != &last->rb_node) + next = rb_entry_rq(rbnext); + } + + return bfq_choose_req(bfqd, next, prev, blk_rq_pos(last)); +} + +/* see the definition of bfq_async_charge_factor for details */ +static unsigned long bfq_serv_to_charge(struct request *rq, + struct bfq_queue *bfqq) +{ + if (bfq_bfqq_sync(bfqq) || bfqq->wr_coeff > 1) + return blk_rq_sectors(rq); + + /* + * If there are no weight-raised queues, then amplify service + * by just the async charge factor; otherwise amplify service + * by twice the async charge factor, to further reduce latency + * for weight-raised queues. + */ + if (bfqq->bfqd->wr_busy_queues == 0) + return blk_rq_sectors(rq) * bfq_async_charge_factor; + + return blk_rq_sectors(rq) * 2 * bfq_async_charge_factor; +} + +/** + * bfq_updated_next_req - update the queue after a new next_rq selection. + * @bfqd: the device data the queue belongs to. + * @bfqq: the queue to update. + * + * If the first request of a queue changes we make sure that the queue + * has enough budget to serve at least its first request (if the + * request has grown). We do this because if the queue has not enough + * budget for its first request, it has to go through two dispatch + * rounds to actually get it dispatched. + */ +static void bfq_updated_next_req(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + struct bfq_service_tree *st = bfq_entity_service_tree(entity); + struct request *next_rq = bfqq->next_rq; + unsigned long new_budget; + + if (!next_rq) + return; + + if (bfqq == bfqd->in_service_queue) + /* + * In order not to break guarantees, budgets cannot be + * changed after an entity has been selected. + */ + return; + + BUG_ON(entity->tree != &st->active); + BUG_ON(entity == entity->sched_data->in_service_entity); + + new_budget = max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(next_rq, bfqq)); + if (entity->budget != new_budget) { + entity->budget = new_budget; + bfq_log_bfqq(bfqd, bfqq, "updated next rq: new budget %lu", + new_budget); + bfq_requeue_bfqq(bfqd, bfqq, false); + } +} + +static unsigned int bfq_wr_duration(struct bfq_data *bfqd) +{ + u64 dur; + + if (bfqd->bfq_wr_max_time > 0) + return bfqd->bfq_wr_max_time; + + dur = bfqd->RT_prod; + do_div(dur, bfqd->peak_rate); + + /* + * Limit duration between 3 and 13 seconds. Tests show that + * higher values than 13 seconds often yield the opposite of + * the desired result, i.e., worsen responsiveness by letting + * non-interactive and non-soft-real-time applications + * preserve weight raising for a too long time interval. + * + * On the other end, lower values than 3 seconds make it + * difficult for most interactive tasks to complete their jobs + * before weight-raising finishes. + */ + if (dur > msecs_to_jiffies(13000)) + dur = msecs_to_jiffies(13000); + else if (dur < msecs_to_jiffies(3000)) + dur = msecs_to_jiffies(3000); + + return dur; +} + +static void +bfq_bfqq_resume_state(struct bfq_queue *bfqq, struct bfq_data *bfqd, + struct bfq_io_cq *bic, bool bfq_already_existing) +{ + unsigned int old_wr_coeff; + bool busy = bfq_already_existing && bfq_bfqq_busy(bfqq); + + if (bic->saved_has_short_ttime) + bfq_mark_bfqq_has_short_ttime(bfqq); + else + bfq_clear_bfqq_has_short_ttime(bfqq); + + if (bic->saved_IO_bound) + bfq_mark_bfqq_IO_bound(bfqq); + else + bfq_clear_bfqq_IO_bound(bfqq); + + if (unlikely(busy)) + old_wr_coeff = bfqq->wr_coeff; + + bfqq->ttime = bic->saved_ttime; + bfqq->wr_coeff = bic->saved_wr_coeff; + bfqq->wr_start_at_switch_to_srt = bic->saved_wr_start_at_switch_to_srt; + BUG_ON(time_is_after_jiffies(bfqq->wr_start_at_switch_to_srt)); + bfqq->last_wr_start_finish = bic->saved_last_wr_start_finish; + bfqq->wr_cur_max_time = bic->saved_wr_cur_max_time; + BUG_ON(time_is_after_jiffies(bfqq->last_wr_start_finish)); + + if (bfqq->wr_coeff > 1 && (bfq_bfqq_in_large_burst(bfqq) || + time_is_before_jiffies(bfqq->last_wr_start_finish + + bfqq->wr_cur_max_time))) { + bfq_log_bfqq(bfqq->bfqd, bfqq, + "resume state: switching off wr (%lu + %lu < %lu)", + bfqq->last_wr_start_finish, bfqq->wr_cur_max_time, + jiffies); + + bfqq->wr_coeff = 1; + } + + /* make sure weight will be updated, however we got here */ + bfqq->entity.prio_changed = 1; + + if (likely(!busy)) + return; + + if (old_wr_coeff == 1 && bfqq->wr_coeff > 1) { + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + } else if (old_wr_coeff > 1 && bfqq->wr_coeff == 1) { + bfqd->wr_busy_queues--; + BUG_ON(bfqd->wr_busy_queues < 0); + } +} + +static int bfqq_process_refs(struct bfq_queue *bfqq) +{ + int process_refs, io_refs; + + lockdep_assert_held(&bfqq->bfqd->lock); + + io_refs = bfqq->allocated; + process_refs = bfqq->ref - io_refs - bfqq->entity.on_st; + BUG_ON(process_refs < 0); + return process_refs; +} + +/* Empty burst list and add just bfqq (see comments to bfq_handle_burst) */ +static void bfq_reset_burst_list(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct bfq_queue *item; + struct hlist_node *n; + + hlist_for_each_entry_safe(item, n, &bfqd->burst_list, burst_list_node) + hlist_del_init(&item->burst_list_node); + hlist_add_head(&bfqq->burst_list_node, &bfqd->burst_list); + bfqd->burst_size = 1; + bfqd->burst_parent_entity = bfqq->entity.parent; +} + +/* Add bfqq to the list of queues in current burst (see bfq_handle_burst) */ +static void bfq_add_to_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + /* Increment burst size to take into account also bfqq */ + bfqd->burst_size++; + + bfq_log_bfqq(bfqd, bfqq, "add_to_burst %d", bfqd->burst_size); + + BUG_ON(bfqd->burst_size > bfqd->bfq_large_burst_thresh); + + if (bfqd->burst_size == bfqd->bfq_large_burst_thresh) { + struct bfq_queue *pos, *bfqq_item; + struct hlist_node *n; + + /* + * Enough queues have been activated shortly after each + * other to consider this burst as large. + */ + bfqd->large_burst = true; + bfq_log_bfqq(bfqd, bfqq, "add_to_burst: large burst started"); + + /* + * We can now mark all queues in the burst list as + * belonging to a large burst. + */ + hlist_for_each_entry(bfqq_item, &bfqd->burst_list, + burst_list_node) { + bfq_mark_bfqq_in_large_burst(bfqq_item); + bfq_log_bfqq(bfqd, bfqq_item, "marked in large burst"); + } + bfq_mark_bfqq_in_large_burst(bfqq); + bfq_log_bfqq(bfqd, bfqq, "marked in large burst"); + + /* + * From now on, and until the current burst finishes, any + * new queue being activated shortly after the last queue + * was inserted in the burst can be immediately marked as + * belonging to a large burst. So the burst list is not + * needed any more. Remove it. + */ + hlist_for_each_entry_safe(pos, n, &bfqd->burst_list, + burst_list_node) + hlist_del_init(&pos->burst_list_node); + } else /* + * Burst not yet large: add bfqq to the burst list. Do + * not increment the ref counter for bfqq, because bfqq + * is removed from the burst list before freeing bfqq + * in put_queue. + */ + hlist_add_head(&bfqq->burst_list_node, &bfqd->burst_list); +} + +/* + * If many queues belonging to the same group happen to be created + * shortly after each other, then the processes associated with these + * queues have typically a common goal. In particular, bursts of queue + * creations are usually caused by services or applications that spawn + * many parallel threads/processes. Examples are systemd during boot, + * or git grep. To help these processes get their job done as soon as + * possible, it is usually better to not grant either weight-raising + * or device idling to their queues. + * + * In this comment we describe, firstly, the reasons why this fact + * holds, and, secondly, the next function, which implements the main + * steps needed to properly mark these queues so that they can then be + * treated in a different way. + * + * The above services or applications benefit mostly from a high + * throughput: the quicker the requests of the activated queues are + * cumulatively served, the sooner the target job of these queues gets + * completed. As a consequence, weight-raising any of these queues, + * which also implies idling the device for it, is almost always + * counterproductive. In most cases it just lowers throughput. + * + * On the other hand, a burst of queue creations may be caused also by + * the start of an application that does not consist of a lot of + * parallel I/O-bound threads. In fact, with a complex application, + * several short processes may need to be executed to start-up the + * application. In this respect, to start an application as quickly as + * possible, the best thing to do is in any case to privilege the I/O + * related to the application with respect to all other + * I/O. Therefore, the best strategy to start as quickly as possible + * an application that causes a burst of queue creations is to + * weight-raise all the queues created during the burst. This is the + * exact opposite of the best strategy for the other type of bursts. + * + * In the end, to take the best action for each of the two cases, the + * two types of bursts need to be distinguished. Fortunately, this + * seems relatively easy, by looking at the sizes of the bursts. In + * particular, we found a threshold such that only bursts with a + * larger size than that threshold are apparently caused by + * services or commands such as systemd or git grep. For brevity, + * hereafter we call just 'large' these bursts. BFQ *does not* + * weight-raise queues whose creation occurs in a large burst. In + * addition, for each of these queues BFQ performs or does not perform + * idling depending on which choice boosts the throughput more. The + * exact choice depends on the device and request pattern at + * hand. + * + * Unfortunately, false positives may occur while an interactive task + * is starting (e.g., an application is being started). The + * consequence is that the queues associated with the task do not + * enjoy weight raising as expected. Fortunately these false positives + * are very rare. They typically occur if some service happens to + * start doing I/O exactly when the interactive task starts. + * + * Turning back to the next function, it implements all the steps + * needed to detect the occurrence of a large burst and to properly + * mark all the queues belonging to it (so that they can then be + * treated in a different way). This goal is achieved by maintaining a + * "burst list" that holds, temporarily, the queues that belong to the + * burst in progress. The list is then used to mark these queues as + * belonging to a large burst if the burst does become large. The main + * steps are the following. + * + * . when the very first queue is created, the queue is inserted into the + * list (as it could be the first queue in a possible burst) + * + * . if the current burst has not yet become large, and a queue Q that does + * not yet belong to the burst is activated shortly after the last time + * at which a new queue entered the burst list, then the function appends + * Q to the burst list + * + * . if, as a consequence of the previous step, the burst size reaches + * the large-burst threshold, then + * + * . all the queues in the burst list are marked as belonging to a + * large burst + * + * . the burst list is deleted; in fact, the burst list already served + * its purpose (keeping temporarily track of the queues in a burst, + * so as to be able to mark them as belonging to a large burst in the + * previous sub-step), and now is not needed any more + * + * . the device enters a large-burst mode + * + * . if a queue Q that does not belong to the burst is created while + * the device is in large-burst mode and shortly after the last time + * at which a queue either entered the burst list or was marked as + * belonging to the current large burst, then Q is immediately marked + * as belonging to a large burst. + * + * . if a queue Q that does not belong to the burst is created a while + * later, i.e., not shortly after, than the last time at which a queue + * either entered the burst list or was marked as belonging to the + * current large burst, then the current burst is deemed as finished and: + * + * . the large-burst mode is reset if set + * + * . the burst list is emptied + * + * . Q is inserted in the burst list, as Q may be the first queue + * in a possible new burst (then the burst list contains just Q + * after this step). + */ +static void bfq_handle_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + /* + * If bfqq is already in the burst list or is part of a large + * burst, or finally has just been split, then there is + * nothing else to do. + */ + if (!hlist_unhashed(&bfqq->burst_list_node) || + bfq_bfqq_in_large_burst(bfqq) || + time_is_after_eq_jiffies(bfqq->split_time + + msecs_to_jiffies(10))) + return; + + /* + * If bfqq's creation happens late enough, or bfqq belongs to + * a different group than the burst group, then the current + * burst is finished, and related data structures must be + * reset. + * + * In this respect, consider the special case where bfqq is + * the very first queue created after BFQ is selected for this + * device. In this case, last_ins_in_burst and + * burst_parent_entity are not yet significant when we get + * here. But it is easy to verify that, whether or not the + * following condition is true, bfqq will end up being + * inserted into the burst list. In particular the list will + * happen to contain only bfqq. And this is exactly what has + * to happen, as bfqq may be the first queue of the first + * burst. + */ + if (time_is_before_jiffies(bfqd->last_ins_in_burst + + bfqd->bfq_burst_interval) || + bfqq->entity.parent != bfqd->burst_parent_entity) { + bfqd->large_burst = false; + bfq_reset_burst_list(bfqd, bfqq); + bfq_log_bfqq(bfqd, bfqq, + "handle_burst: late activation or different group"); + goto end; + } + + /* + * If we get here, then bfqq is being activated shortly after the + * last queue. So, if the current burst is also large, we can mark + * bfqq as belonging to this large burst immediately. + */ + if (bfqd->large_burst) { + bfq_log_bfqq(bfqd, bfqq, "handle_burst: marked in burst"); + bfq_mark_bfqq_in_large_burst(bfqq); + goto end; + } + + /* + * If we get here, then a large-burst state has not yet been + * reached, but bfqq is being activated shortly after the last + * queue. Then we add bfqq to the burst. + */ + bfq_add_to_burst(bfqd, bfqq); +end: + /* + * At this point, bfqq either has been added to the current + * burst or has caused the current burst to terminate and a + * possible new burst to start. In particular, in the second + * case, bfqq has become the first queue in the possible new + * burst. In both cases last_ins_in_burst needs to be moved + * forward. + */ + bfqd->last_ins_in_burst = jiffies; + +} + +static int bfq_bfqq_budget_left(struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + + return entity->budget - entity->service; +} + +/* + * If enough samples have been computed, return the current max budget + * stored in bfqd, which is dynamically updated according to the + * estimated disk peak rate; otherwise return the default max budget + */ +static int bfq_max_budget(struct bfq_data *bfqd) +{ + if (bfqd->budgets_assigned < bfq_stats_min_budgets) + return bfq_default_max_budget; + else + return bfqd->bfq_max_budget; +} + +/* + * Return min budget, which is a fraction of the current or default + * max budget (trying with 1/32) + */ +static int bfq_min_budget(struct bfq_data *bfqd) +{ + if (bfqd->budgets_assigned < bfq_stats_min_budgets) + return bfq_default_max_budget / 32; + else + return bfqd->bfq_max_budget / 32; +} + +static void bfq_bfqq_expire(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool compensate, + enum bfqq_expiration reason); + +/* + * The next function, invoked after the input queue bfqq switches from + * idle to busy, updates the budget of bfqq. The function also tells + * whether the in-service queue should be expired, by returning + * true. The purpose of expiring the in-service queue is to give bfqq + * the chance to possibly preempt the in-service queue, and the reason + * for preempting the in-service queue is to achieve one of the two + * goals below. + * + * 1. Guarantee to bfqq its reserved bandwidth even if bfqq has + * expired because it has remained idle. In particular, bfqq may have + * expired for one of the following two reasons: + * + * - BFQ_BFQQ_NO_MORE_REQUEST bfqq did not enjoy any device idling and + * did not make it to issue a new request before its last request + * was served; + * + * - BFQ_BFQQ_TOO_IDLE bfqq did enjoy device idling, but did not issue + * a new request before the expiration of the idling-time. + * + * Even if bfqq has expired for one of the above reasons, the process + * associated with the queue may be however issuing requests greedily, + * and thus be sensitive to the bandwidth it receives (bfqq may have + * remained idle for other reasons: CPU high load, bfqq not enjoying + * idling, I/O throttling somewhere in the path from the process to + * the I/O scheduler, ...). But if, after every expiration for one of + * the above two reasons, bfqq has to wait for the service of at least + * one full budget of another queue before being served again, then + * bfqq is likely to get a much lower bandwidth or resource time than + * its reserved ones. To address this issue, two countermeasures need + * to be taken. + * + * First, the budget and the timestamps of bfqq need to be updated in + * a special way on bfqq reactivation: they need to be updated as if + * bfqq did not remain idle and did not expire. In fact, if they are + * computed as if bfqq expired and remained idle until reactivation, + * then the process associated with bfqq is treated as if, instead of + * being greedy, it stopped issuing requests when bfqq remained idle, + * and restarts issuing requests only on this reactivation. In other + * words, the scheduler does not help the process recover the "service + * hole" between bfqq expiration and reactivation. As a consequence, + * the process receives a lower bandwidth than its reserved one. In + * contrast, to recover this hole, the budget must be updated as if + * bfqq was not expired at all before this reactivation, i.e., it must + * be set to the value of the remaining budget when bfqq was + * expired. Along the same line, timestamps need to be assigned the + * value they had the last time bfqq was selected for service, i.e., + * before last expiration. Thus timestamps need to be back-shifted + * with respect to their normal computation (see [1] for more details + * on this tricky aspect). + * + * Secondly, to allow the process to recover the hole, the in-service + * queue must be expired too, to give bfqq the chance to preempt it + * immediately. In fact, if bfqq has to wait for a full budget of the + * in-service queue to be completed, then it may become impossible to + * let the process recover the hole, even if the back-shifted + * timestamps of bfqq are lower than those of the in-service queue. If + * this happens for most or all of the holes, then the process may not + * receive its reserved bandwidth. In this respect, it is worth noting + * that, being the service of outstanding requests unpreemptible, a + * little fraction of the holes may however be unrecoverable, thereby + * causing a little loss of bandwidth. + * + * The last important point is detecting whether bfqq does need this + * bandwidth recovery. In this respect, the next function deems the + * process associated with bfqq greedy, and thus allows it to recover + * the hole, if: 1) the process is waiting for the arrival of a new + * request (which implies that bfqq expired for one of the above two + * reasons), and 2) such a request has arrived soon. The first + * condition is controlled through the flag non_blocking_wait_rq, + * while the second through the flag arrived_in_time. If both + * conditions hold, then the function computes the budget in the + * above-described special way, and signals that the in-service queue + * should be expired. Timestamp back-shifting is done later in + * __bfq_activate_entity. + * + * 2. Reduce latency. Even if timestamps are not backshifted to let + * the process associated with bfqq recover a service hole, bfqq may + * however happen to have, after being (re)activated, a lower finish + * timestamp than the in-service queue. That is, the next budget of + * bfqq may have to be completed before the one of the in-service + * queue. If this is the case, then preempting the in-service queue + * allows this goal to be achieved, apart from the unpreemptible, + * outstanding requests mentioned above. + * + * Unfortunately, regardless of which of the above two goals one wants + * to achieve, service trees need first to be updated to know whether + * the in-service queue must be preempted. To have service trees + * correctly updated, the in-service queue must be expired and + * rescheduled, and bfqq must be scheduled too. This is one of the + * most costly operations (in future versions, the scheduling + * mechanism may be re-designed in such a way to make it possible to + * know whether preemption is needed without needing to update service + * trees). In addition, queue preemptions almost always cause random + * I/O, and thus loss of throughput. Because of these facts, the next + * function adopts the following simple scheme to avoid both costly + * operations and too frequent preemptions: it requests the expiration + * of the in-service queue (unconditionally) only for queues that need + * to recover a hole, or that either are weight-raised or deserve to + * be weight-raised. + */ +static bool bfq_bfqq_update_budg_for_activation(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool arrived_in_time, + bool wr_or_deserves_wr) +{ + struct bfq_entity *entity = &bfqq->entity; + + if (bfq_bfqq_non_blocking_wait_rq(bfqq) && arrived_in_time) { + /* + * We do not clear the flag non_blocking_wait_rq here, as + * the latter is used in bfq_activate_bfqq to signal + * that timestamps need to be back-shifted (and is + * cleared right after). + */ + + /* + * In next assignment we rely on that either + * entity->service or entity->budget are not updated + * on expiration if bfqq is empty (see + * __bfq_bfqq_recalc_budget). Thus both quantities + * remain unchanged after such an expiration, and the + * following statement therefore assigns to + * entity->budget the remaining budget on such an + * expiration. For clarity, entity->service is not + * updated on expiration in any case, and, in normal + * operation, is reset only when bfqq is selected for + * service (see bfq_get_next_queue). + */ + BUG_ON(bfqq->max_budget < 0); + entity->budget = min_t(unsigned long, + bfq_bfqq_budget_left(bfqq), + bfqq->max_budget); + + BUG_ON(entity->budget < 0); + return true; + } + + BUG_ON(bfqq->max_budget < 0); + entity->budget = max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(bfqq->next_rq, bfqq)); + BUG_ON(entity->budget < 0); + + bfq_clear_bfqq_non_blocking_wait_rq(bfqq); + return wr_or_deserves_wr; +} + +/* + * Return the farthest future time instant according to jiffies + * macros. + */ +static unsigned long bfq_greatest_from_now(void) +{ + return jiffies + MAX_JIFFY_OFFSET; +} + +/* + * Return the farthest past time instant according to jiffies + * macros. + */ +static unsigned long bfq_smallest_from_now(void) +{ + return jiffies - MAX_JIFFY_OFFSET; +} + +static void bfq_update_bfqq_wr_on_rq_arrival(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + unsigned int old_wr_coeff, + bool wr_or_deserves_wr, + bool interactive, + bool in_burst, + bool soft_rt) +{ + if (old_wr_coeff == 1 && wr_or_deserves_wr) { + /* start a weight-raising period */ + if (interactive) { + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + } else { + /* + * No interactive weight raising in progress + * here: assign minus infinity to + * wr_start_at_switch_to_srt, to make sure + * that, at the end of the soft-real-time + * weight raising periods that is starting + * now, no interactive weight-raising period + * may be wrongly considered as still in + * progress (and thus actually started by + * mistake). + */ + bfqq->wr_start_at_switch_to_srt = + bfq_smallest_from_now(); + bfqq->wr_coeff = bfqd->bfq_wr_coeff * + BFQ_SOFTRT_WEIGHT_FACTOR; + bfqq->wr_cur_max_time = + bfqd->bfq_wr_rt_max_time; + } + /* + * If needed, further reduce budget to make sure it is + * close to bfqq's backlog, so as to reduce the + * scheduling-error component due to a too large + * budget. Do not care about throughput consequences, + * but only about latency. Finally, do not assign a + * too small budget either, to avoid increasing + * latency by causing too frequent expirations. + */ + bfqq->entity.budget = min_t(unsigned long, + bfqq->entity.budget, + 2 * bfq_min_budget(bfqd)); + + bfq_log_bfqq(bfqd, bfqq, + "wrais starting at %lu, rais_max_time %u", + jiffies, + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } else if (old_wr_coeff > 1) { + if (interactive) { /* update wr coeff and duration */ + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + } else if (in_burst) { + bfqq->wr_coeff = 1; + bfq_log_bfqq(bfqd, bfqq, + "wrais ending at %lu, rais_max_time %u", + jiffies, + jiffies_to_msecs(bfqq-> + wr_cur_max_time)); + } else if (soft_rt) { + /* + * The application is now or still meeting the + * requirements for being deemed soft rt. We + * can then correctly and safely (re)charge + * the weight-raising duration for the + * application with the weight-raising + * duration for soft rt applications. + * + * In particular, doing this recharge now, i.e., + * before the weight-raising period for the + * application finishes, reduces the probability + * of the following negative scenario: + * 1) the weight of a soft rt application is + * raised at startup (as for any newly + * created application), + * 2) since the application is not interactive, + * at a certain time weight-raising is + * stopped for the application, + * 3) at that time the application happens to + * still have pending requests, and hence + * is destined to not have a chance to be + * deemed soft rt before these requests are + * completed (see the comments to the + * function bfq_bfqq_softrt_next_start() + * for details on soft rt detection), + * 4) these pending requests experience a high + * latency because the application is not + * weight-raised while they are pending. + */ + if (bfqq->wr_cur_max_time != + bfqd->bfq_wr_rt_max_time) { + bfqq->wr_start_at_switch_to_srt = + bfqq->last_wr_start_finish; + BUG_ON(time_is_after_jiffies(bfqq->last_wr_start_finish)); + + bfqq->wr_cur_max_time = + bfqd->bfq_wr_rt_max_time; + bfqq->wr_coeff = bfqd->bfq_wr_coeff * + BFQ_SOFTRT_WEIGHT_FACTOR; + bfq_log_bfqq(bfqd, bfqq, + "switching to soft_rt wr"); + } else + bfq_log_bfqq(bfqd, bfqq, + "moving forward soft_rt wr duration"); + bfqq->last_wr_start_finish = jiffies; + } + } +} + +static bool bfq_bfqq_idle_for_long_time(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + return bfqq->dispatched == 0 && + time_is_before_jiffies( + bfqq->budget_timeout + + bfqd->bfq_wr_min_idle_time); +} + +static void bfq_bfqq_handle_idle_busy_switch(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + int old_wr_coeff, + struct request *rq, + bool *interactive) +{ + bool soft_rt, in_burst, wr_or_deserves_wr, + bfqq_wants_to_preempt, + idle_for_long_time = bfq_bfqq_idle_for_long_time(bfqd, bfqq), + /* + * See the comments on + * bfq_bfqq_update_budg_for_activation for + * details on the usage of the next variable. + */ + arrived_in_time = ktime_get_ns() <= + bfqq->ttime.last_end_request + + bfqd->bfq_slice_idle * 3; + + bfq_log_bfqq(bfqd, bfqq, + "bfq_add_request non-busy: " + "jiffies %lu, in_time %d, idle_long %d busyw %d " + "wr_coeff %u", + jiffies, arrived_in_time, + idle_for_long_time, + bfq_bfqq_non_blocking_wait_rq(bfqq), + old_wr_coeff); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + BUG_ON(bfqq == bfqd->in_service_queue); + bfqg_stats_update_io_add(bfqq_group(RQ_BFQQ(rq)), bfqq, rq->cmd_flags); + + /* + * bfqq deserves to be weight-raised if: + * - it is sync, + * - it does not belong to a large burst, + * - it has been idle for enough time or is soft real-time, + * - is linked to a bfq_io_cq (it is not shared in any sense) + */ + in_burst = bfq_bfqq_in_large_burst(bfqq); + soft_rt = bfqd->bfq_wr_max_softrt_rate > 0 && + !in_burst && + time_is_before_jiffies(bfqq->soft_rt_next_start); + *interactive = + !in_burst && + idle_for_long_time; + wr_or_deserves_wr = bfqd->low_latency && + (bfqq->wr_coeff > 1 || + (bfq_bfqq_sync(bfqq) && + bfqq->bic && (*interactive || soft_rt))); + + bfq_log_bfqq(bfqd, bfqq, + "bfq_add_request: " + "in_burst %d, " + "soft_rt %d (next %lu), inter %d, bic %p", + bfq_bfqq_in_large_burst(bfqq), soft_rt, + bfqq->soft_rt_next_start, + *interactive, + bfqq->bic); + + /* + * Using the last flag, update budget and check whether bfqq + * may want to preempt the in-service queue. + */ + bfqq_wants_to_preempt = + bfq_bfqq_update_budg_for_activation(bfqd, bfqq, + arrived_in_time, + wr_or_deserves_wr); + + /* + * If bfqq happened to be activated in a burst, but has been + * idle for much more than an interactive queue, then we + * assume that, in the overall I/O initiated in the burst, the + * I/O associated with bfqq is finished. So bfqq does not need + * to be treated as a queue belonging to a burst + * anymore. Accordingly, we reset bfqq's in_large_burst flag + * if set, and remove bfqq from the burst list if it's + * there. We do not decrement burst_size, because the fact + * that bfqq does not need to belong to the burst list any + * more does not invalidate the fact that bfqq was created in + * a burst. + */ + if (likely(!bfq_bfqq_just_created(bfqq)) && + idle_for_long_time && + time_is_before_jiffies( + bfqq->budget_timeout + + msecs_to_jiffies(10000))) { + hlist_del_init(&bfqq->burst_list_node); + bfq_clear_bfqq_in_large_burst(bfqq); + } + + bfq_clear_bfqq_just_created(bfqq); + + if (!bfq_bfqq_IO_bound(bfqq)) { + if (arrived_in_time) { + bfqq->requests_within_timer++; + if (bfqq->requests_within_timer >= + bfqd->bfq_requests_within_timer) + bfq_mark_bfqq_IO_bound(bfqq); + } else + bfqq->requests_within_timer = 0; + bfq_log_bfqq(bfqd, bfqq, "requests in time %d", + bfqq->requests_within_timer); + } + + if (bfqd->low_latency) { + if (unlikely(time_is_after_jiffies(bfqq->split_time))) + /* wraparound */ + bfqq->split_time = + jiffies - bfqd->bfq_wr_min_idle_time - 1; + + if (time_is_before_jiffies(bfqq->split_time + + bfqd->bfq_wr_min_idle_time)) { + bfq_update_bfqq_wr_on_rq_arrival(bfqd, bfqq, + old_wr_coeff, + wr_or_deserves_wr, + *interactive, + in_burst, + soft_rt); + + if (old_wr_coeff != bfqq->wr_coeff) + bfqq->entity.prio_changed = 1; + } + } + + bfqq->last_idle_bklogged = jiffies; + bfqq->service_from_backlogged = 0; + bfq_clear_bfqq_softrt_update(bfqq); + + bfq_add_bfqq_busy(bfqd, bfqq); + + /* + * Expire in-service queue only if preemption may be needed + * for guarantees. In this respect, the function + * next_queue_may_preempt just checks a simple, necessary + * condition, and not a sufficient condition based on + * timestamps. In fact, for the latter condition to be + * evaluated, timestamps would need first to be updated, and + * this operation is quite costly (see the comments on the + * function bfq_bfqq_update_budg_for_activation). + */ + if (bfqd->in_service_queue && bfqq_wants_to_preempt && + bfqd->in_service_queue->wr_coeff < bfqq->wr_coeff && + next_queue_may_preempt(bfqd)) { + struct bfq_queue *in_serv = + bfqd->in_service_queue; + BUG_ON(in_serv == bfqq); + + bfq_bfqq_expire(bfqd, bfqd->in_service_queue, + false, BFQ_BFQQ_PREEMPTED); + } +} + +static void bfq_add_request(struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + struct bfq_data *bfqd = bfqq->bfqd; + struct request *next_rq, *prev; + unsigned int old_wr_coeff = bfqq->wr_coeff; + bool interactive = false; + + bfq_log_bfqq(bfqd, bfqq, "add_request: size %u %s", + blk_rq_sectors(rq), rq_is_sync(rq) ? "S" : "A"); + + if (bfqq->wr_coeff > 1) /* queue is being weight-raised */ + bfq_log_bfqq(bfqd, bfqq, + "raising period dur %u/%u msec, old coeff %u, w %d(%d)", + jiffies_to_msecs(jiffies - bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time), + bfqq->wr_coeff, + bfqq->entity.weight, bfqq->entity.orig_weight); + + bfqq->queued[rq_is_sync(rq)]++; + bfqd->queued++; + + BUG_ON(!RQ_BFQQ(rq)); + BUG_ON(RQ_BFQQ(rq) != bfqq); + elv_rb_add(&bfqq->sort_list, rq); + + /* + * Check if this request is a better next-to-serve candidate. + */ + prev = bfqq->next_rq; + next_rq = bfq_choose_req(bfqd, bfqq->next_rq, rq, bfqd->last_position); + BUG_ON(!next_rq); + BUG_ON(!RQ_BFQQ(next_rq)); + BUG_ON(RQ_BFQQ(next_rq) != bfqq); + bfqq->next_rq = next_rq; + + /* + * Adjust priority tree position, if next_rq changes. + */ + if (prev != bfqq->next_rq) + bfq_pos_tree_add_move(bfqd, bfqq); + + if (!bfq_bfqq_busy(bfqq)) /* switching to busy ... */ + bfq_bfqq_handle_idle_busy_switch(bfqd, bfqq, old_wr_coeff, + rq, &interactive); + else { + if (bfqd->low_latency && old_wr_coeff == 1 && !rq_is_sync(rq) && + time_is_before_jiffies( + bfqq->last_wr_start_finish + + bfqd->bfq_wr_min_inter_arr_async)) { + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqd, bfqq, + "non-idle wrais starting, " + "wr_max_time %u wr_busy %d", + jiffies_to_msecs(bfqq->wr_cur_max_time), + bfqd->wr_busy_queues); + } + if (prev != bfqq->next_rq) + bfq_updated_next_req(bfqd, bfqq); + } + + /* + * Assign jiffies to last_wr_start_finish in the following + * cases: + * + * . if bfqq is not going to be weight-raised, because, for + * non weight-raised queues, last_wr_start_finish stores the + * arrival time of the last request; as of now, this piece + * of information is used only for deciding whether to + * weight-raise async queues + * + * . if bfqq is not weight-raised, because, if bfqq is now + * switching to weight-raised, then last_wr_start_finish + * stores the time when weight-raising starts + * + * . if bfqq is interactive, because, regardless of whether + * bfqq is currently weight-raised, the weight-raising + * period must start or restart (this case is considered + * separately because it is not detected by the above + * conditions, if bfqq is already weight-raised) + * + * last_wr_start_finish has to be updated also if bfqq is soft + * real-time, because the weight-raising period is constantly + * restarted on idle-to-busy transitions for these queues, but + * this is already done in bfq_bfqq_handle_idle_busy_switch if + * needed. + */ + if (bfqd->low_latency && + (old_wr_coeff == 1 || bfqq->wr_coeff == 1 || interactive)) + bfqq->last_wr_start_finish = jiffies; +} + +static struct request *bfq_find_rq_fmerge(struct bfq_data *bfqd, + struct bio *bio, + struct request_queue *q) +{ + struct bfq_queue *bfqq = bfqd->bio_bfqq; + + BUG_ON(!bfqd->bio_bfqq_set); + + if (bfqq) + return elv_rb_find(&bfqq->sort_list, bio_end_sector(bio)); + + return NULL; +} + +static sector_t get_sdist(sector_t last_pos, struct request *rq) +{ + sector_t sdist = 0; + + if (last_pos) { + if (last_pos < blk_rq_pos(rq)) + sdist = blk_rq_pos(rq) - last_pos; + else + sdist = last_pos - blk_rq_pos(rq); + } + + return sdist; +} + +#if 0 /* Still not clear if we can do without next two functions */ +static void bfq_activate_request(struct request_queue *q, struct request *rq) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + bfqd->rq_in_driver++; +} + +static void bfq_deactivate_request(struct request_queue *q, struct request *rq) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + + BUG_ON(bfqd->rq_in_driver == 0); + bfqd->rq_in_driver--; +} +#endif + +static void bfq_remove_request(struct request_queue *q, + struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + struct bfq_data *bfqd = bfqq->bfqd; + const int sync = rq_is_sync(rq); + + BUG_ON(bfqq->entity.service > bfqq->entity.budget && + bfqq == bfqd->in_service_queue); + + if (bfqq->next_rq == rq) { + bfqq->next_rq = bfq_find_next_rq(bfqd, bfqq, rq); + if (bfqq->next_rq && !RQ_BFQQ(bfqq->next_rq)) { + pr_crit("no bfqq! for next rq %p bfqq %p\n", + bfqq->next_rq, bfqq); + } + + BUG_ON(bfqq->next_rq && !RQ_BFQQ(bfqq->next_rq)); + if (bfqq->next_rq && RQ_BFQQ(bfqq->next_rq) != bfqq) { + pr_crit( + "wrong bfqq! for next rq %p, rq_bfqq %p bfqq %p\n", + bfqq->next_rq, RQ_BFQQ(bfqq->next_rq), bfqq); + } + BUG_ON(bfqq->next_rq && RQ_BFQQ(bfqq->next_rq) != bfqq); + + bfq_updated_next_req(bfqd, bfqq); + } + + if (rq->queuelist.prev != &rq->queuelist) + list_del_init(&rq->queuelist); + BUG_ON(bfqq->queued[sync] == 0); + bfqq->queued[sync]--; + bfqd->queued--; + elv_rb_del(&bfqq->sort_list, rq); + + elv_rqhash_del(q, rq); + if (q->last_merge == rq) + q->last_merge = NULL; + + if (RB_EMPTY_ROOT(&bfqq->sort_list)) { + bfqq->next_rq = NULL; + + BUG_ON(bfqq->entity.budget < 0); + + if (bfq_bfqq_busy(bfqq) && bfqq != bfqd->in_service_queue) { + BUG_ON(bfqq->ref < 2); /* referred by rq and on tree */ + bfq_del_bfqq_busy(bfqd, bfqq, false); + /* + * bfqq emptied. In normal operation, when + * bfqq is empty, bfqq->entity.service and + * bfqq->entity.budget must contain, + * respectively, the service received and the + * budget used last time bfqq emptied. These + * facts do not hold in this case, as at least + * this last removal occurred while bfqq is + * not in service. To avoid inconsistencies, + * reset both bfqq->entity.service and + * bfqq->entity.budget, if bfqq has still a + * process that may issue I/O requests to it. + */ + bfqq->entity.budget = bfqq->entity.service = 0; + } + + /* + * Remove queue from request-position tree as it is empty. + */ + if (bfqq->pos_root) { + rb_erase(&bfqq->pos_node, bfqq->pos_root); + bfqq->pos_root = NULL; + } + } + + if (rq->cmd_flags & REQ_META) { + BUG_ON(bfqq->meta_pending == 0); + bfqq->meta_pending--; + } + bfqg_stats_update_io_remove(bfqq_group(bfqq), rq->cmd_flags); +} + +static bool bfq_bio_merge(struct blk_mq_hw_ctx *hctx, struct bio *bio) +{ + struct request_queue *q = hctx->queue; + struct bfq_data *bfqd = q->elevator->elevator_data; + struct request *free = NULL; + /* + * bfq_bic_lookup grabs the queue_lock: invoke it now and + * store its return value for later use, to avoid nesting + * queue_lock inside the bfqd->lock. We assume that the bic + * returned by bfq_bic_lookup does not go away before + * bfqd->lock is taken. + */ + struct bfq_io_cq *bic = bfq_bic_lookup(bfqd, current->io_context, q); + bool ret; + + spin_lock_irq(&bfqd->lock); + + if (bic) + bfqd->bio_bfqq = bic_to_bfqq(bic, op_is_sync(bio->bi_opf)); + else + bfqd->bio_bfqq = NULL; + bfqd->bio_bic = bic; + /* Set next flag just for testing purposes */ + bfqd->bio_bfqq_set = true; + + ret = blk_mq_sched_try_merge(q, bio, &free); + + /* + * XXX Not yet freeing without lock held, to avoid an + * inconsistency with respect to the lock-protected invocation + * of blk_mq_sched_try_insert_merge in bfq_bio_merge. Waiting + * for clarifications from Jens. + */ + if (free) + blk_mq_free_request(free); + bfqd->bio_bfqq_set = false; + spin_unlock_irq(&bfqd->lock); + + return ret; +} + +static int bfq_request_merge(struct request_queue *q, struct request **req, + struct bio *bio) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + struct request *__rq; + + __rq = bfq_find_rq_fmerge(bfqd, bio, q); + if (__rq && elv_bio_merge_ok(__rq, bio)) { + *req = __rq; + bfq_log(bfqd, "request_merge: req %p", __rq); + + return ELEVATOR_FRONT_MERGE; + } + + return ELEVATOR_NO_MERGE; +} + +static void bfq_request_merged(struct request_queue *q, struct request *req, + enum elv_merge type) +{ + BUG_ON(req->rq_flags & RQF_DISP_LIST); + + if (type == ELEVATOR_FRONT_MERGE && + rb_prev(&req->rb_node) && + blk_rq_pos(req) < + blk_rq_pos(container_of(rb_prev(&req->rb_node), + struct request, rb_node))) { + struct bfq_queue *bfqq = RQ_BFQQ(req); + struct bfq_data *bfqd = bfqq->bfqd; + struct request *prev, *next_rq; + + /* Reposition request in its sort_list */ + elv_rb_del(&bfqq->sort_list, req); + BUG_ON(!RQ_BFQQ(req)); + BUG_ON(RQ_BFQQ(req) != bfqq); + elv_rb_add(&bfqq->sort_list, req); + + /* Choose next request to be served for bfqq */ + prev = bfqq->next_rq; + next_rq = bfq_choose_req(bfqd, bfqq->next_rq, req, + bfqd->last_position); + BUG_ON(!next_rq); + + bfqq->next_rq = next_rq; + + bfq_log_bfqq(bfqd, bfqq, + "requests_merged: req %p prev %p next_rq %p bfqq %p", + req, prev, next_rq, bfqq); + + /* + * If next_rq changes, update both the queue's budget to + * fit the new request and the queue's position in its + * rq_pos_tree. + */ + if (prev != bfqq->next_rq) { + bfq_updated_next_req(bfqd, bfqq); + bfq_pos_tree_add_move(bfqd, bfqq); + } + } +} + +static void bfq_requests_merged(struct request_queue *q, struct request *rq, + struct request *next) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq), *next_bfqq = RQ_BFQQ(next); + + BUG_ON(!RQ_BFQQ(rq)); + BUG_ON(!RQ_BFQQ(next)); + BUG_ON(rq->rq_flags & RQF_DISP_LIST); + BUG_ON(next->rq_flags & RQF_DISP_LIST); + + if (!RB_EMPTY_NODE(&rq->rb_node)) + goto end; + + bfq_log_bfqq(bfqq->bfqd, bfqq, + "requests_merged: rq %p next %p bfqq %p next_bfqq %p", + rq, next, bfqq, next_bfqq); + + spin_lock_irq(&bfqq->bfqd->lock); + + /* + * If next and rq belong to the same bfq_queue and next is older + * than rq, then reposition rq in the fifo (by substituting next + * with rq). Otherwise, if next and rq belong to different + * bfq_queues, never reposition rq: in fact, we would have to + * reposition it with respect to next's position in its own fifo, + * which would most certainly be too expensive with respect to + * the benefits. + */ + if (bfqq == next_bfqq && + !list_empty(&rq->queuelist) && !list_empty(&next->queuelist) && + next->fifo_time < rq->fifo_time) { + list_del_init(&rq->queuelist); + list_replace_init(&next->queuelist, &rq->queuelist); + rq->fifo_time = next->fifo_time; + } + + if (bfqq->next_rq == next) + bfqq->next_rq = rq; + + bfq_remove_request(q, next); + + spin_unlock_irq(&bfqq->bfqd->lock); +end: + bfqg_stats_update_io_merged(bfqq_group(bfqq), next->cmd_flags); +} + +/* Must be called with bfqq != NULL */ +static void bfq_bfqq_end_wr(struct bfq_queue *bfqq) +{ + BUG_ON(!bfqq); + + if (bfq_bfqq_busy(bfqq)) { + bfqq->bfqd->wr_busy_queues--; + BUG_ON(bfqq->bfqd->wr_busy_queues < 0); + } + bfqq->wr_coeff = 1; + bfqq->wr_cur_max_time = 0; + bfqq->last_wr_start_finish = jiffies; + /* + * Trigger a weight change on the next invocation of + * __bfq_entity_update_weight_prio. + */ + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqq->bfqd, bfqq, + "end_wr: wrais ending at %lu, rais_max_time %u", + bfqq->last_wr_start_finish, + jiffies_to_msecs(bfqq->wr_cur_max_time)); + bfq_log_bfqq(bfqq->bfqd, bfqq, "end_wr: wr_busy %d", + bfqq->bfqd->wr_busy_queues); +} + +static void bfq_end_wr_async_queues(struct bfq_data *bfqd, + struct bfq_group *bfqg) +{ + int i, j; + + for (i = 0; i < 2; i++) + for (j = 0; j < IOPRIO_BE_NR; j++) + if (bfqg->async_bfqq[i][j]) + bfq_bfqq_end_wr(bfqg->async_bfqq[i][j]); + if (bfqg->async_idle_bfqq) + bfq_bfqq_end_wr(bfqg->async_idle_bfqq); +} + +static void bfq_end_wr(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq; + + spin_lock_irq(&bfqd->lock); + + list_for_each_entry(bfqq, &bfqd->active_list, bfqq_list) + bfq_bfqq_end_wr(bfqq); + list_for_each_entry(bfqq, &bfqd->idle_list, bfqq_list) + bfq_bfqq_end_wr(bfqq); + bfq_end_wr_async(bfqd); + + spin_unlock_irq(&bfqd->lock); +} + +static sector_t bfq_io_struct_pos(void *io_struct, bool request) +{ + if (request) + return blk_rq_pos(io_struct); + else + return ((struct bio *)io_struct)->bi_iter.bi_sector; +} + +static int bfq_rq_close_to_sector(void *io_struct, bool request, + sector_t sector) +{ + return abs(bfq_io_struct_pos(io_struct, request) - sector) <= + BFQQ_CLOSE_THR; +} + +static struct bfq_queue *bfqq_find_close(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + sector_t sector) +{ + struct rb_root *root = &bfq_bfqq_to_bfqg(bfqq)->rq_pos_tree; + struct rb_node *parent, *node; + struct bfq_queue *__bfqq; + + if (RB_EMPTY_ROOT(root)) + return NULL; + + /* + * First, if we find a request starting at the end of the last + * request, choose it. + */ + __bfqq = bfq_rq_pos_tree_lookup(bfqd, root, sector, &parent, NULL); + if (__bfqq) + return __bfqq; + + /* + * If the exact sector wasn't found, the parent of the NULL leaf + * will contain the closest sector (rq_pos_tree sorted by + * next_request position). + */ + __bfqq = rb_entry(parent, struct bfq_queue, pos_node); + if (bfq_rq_close_to_sector(__bfqq->next_rq, true, sector)) + return __bfqq; + + if (blk_rq_pos(__bfqq->next_rq) < sector) + node = rb_next(&__bfqq->pos_node); + else + node = rb_prev(&__bfqq->pos_node); + if (!node) + return NULL; + + __bfqq = rb_entry(node, struct bfq_queue, pos_node); + if (bfq_rq_close_to_sector(__bfqq->next_rq, true, sector)) + return __bfqq; + + return NULL; +} + +static struct bfq_queue *bfq_find_close_cooperator(struct bfq_data *bfqd, + struct bfq_queue *cur_bfqq, + sector_t sector) +{ + struct bfq_queue *bfqq; + + /* + * We shall notice if some of the queues are cooperating, + * e.g., working closely on the same area of the device. In + * that case, we can group them together and: 1) don't waste + * time idling, and 2) serve the union of their requests in + * the best possible order for throughput. + */ + bfqq = bfqq_find_close(bfqd, cur_bfqq, sector); + if (!bfqq || bfqq == cur_bfqq) + return NULL; + + return bfqq; +} + +static struct bfq_queue * +bfq_setup_merge(struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) +{ + int process_refs, new_process_refs; + struct bfq_queue *__bfqq; + + /* + * If there are no process references on the new_bfqq, then it is + * unsafe to follow the ->new_bfqq chain as other bfqq's in the chain + * may have dropped their last reference (not just their last process + * reference). + */ + if (!bfqq_process_refs(new_bfqq)) + return NULL; + + /* Avoid a circular list and skip interim queue merges. */ + while ((__bfqq = new_bfqq->new_bfqq)) { + if (__bfqq == bfqq) + return NULL; + new_bfqq = __bfqq; + } + + process_refs = bfqq_process_refs(bfqq); + new_process_refs = bfqq_process_refs(new_bfqq); + /* + * If the process for the bfqq has gone away, there is no + * sense in merging the queues. + */ + if (process_refs == 0 || new_process_refs == 0) + return NULL; + + bfq_log_bfqq(bfqq->bfqd, bfqq, "scheduling merge with queue %d", + new_bfqq->pid); + + /* + * Merging is just a redirection: the requests of the process + * owning one of the two queues are redirected to the other queue. + * The latter queue, in its turn, is set as shared if this is the + * first time that the requests of some process are redirected to + * it. + * + * We redirect bfqq to new_bfqq and not the opposite, because + * we are in the context of the process owning bfqq, thus we + * have the io_cq of this process. So we can immediately + * configure this io_cq to redirect the requests of the + * process to new_bfqq. In contrast, the io_cq of new_bfqq is + * not available any more (new_bfqq->bic == NULL). + * + * Anyway, even in case new_bfqq coincides with the in-service + * queue, redirecting requests the in-service queue is the + * best option, as we feed the in-service queue with new + * requests close to the last request served and, by doing so, + * are likely to increase the throughput. + */ + bfqq->new_bfqq = new_bfqq; + new_bfqq->ref += process_refs; + return new_bfqq; +} + +static bool bfq_may_be_close_cooperator(struct bfq_queue *bfqq, + struct bfq_queue *new_bfqq) +{ + if (bfq_class_idle(bfqq) || bfq_class_idle(new_bfqq) || + (bfqq->ioprio_class != new_bfqq->ioprio_class)) + return false; + + /* + * If either of the queues has already been detected as seeky, + * then merging it with the other queue is unlikely to lead to + * sequential I/O. + */ + if (BFQQ_SEEKY(bfqq) || BFQQ_SEEKY(new_bfqq)) + return false; + + /* + * Interleaved I/O is known to be done by (some) applications + * only for reads, so it does not make sense to merge async + * queues. + */ + if (!bfq_bfqq_sync(bfqq) || !bfq_bfqq_sync(new_bfqq)) + return false; + + return true; +} + +/* + * If this function returns true, then bfqq cannot be merged. The idea + * is that true cooperation happens very early after processes start + * to do I/O. Usually, late cooperations are just accidental false + * positives. In case bfqq is weight-raised, such false positives + * would evidently degrade latency guarantees for bfqq. + */ +static bool wr_from_too_long(struct bfq_queue *bfqq) +{ + return bfqq->wr_coeff > 1 && + time_is_before_jiffies(bfqq->last_wr_start_finish + + msecs_to_jiffies(100)); +} + +/* + * Attempt to schedule a merge of bfqq with the currently in-service + * queue or with a close queue among the scheduled queues. Return + * NULL if no merge was scheduled, a pointer to the shared bfq_queue + * structure otherwise. + * + * The OOM queue is not allowed to participate to cooperation: in fact, since + * the requests temporarily redirected to the OOM queue could be redirected + * again to dedicated queues at any time, the state needed to correctly + * handle merging with the OOM queue would be quite complex and expensive + * to maintain. Besides, in such a critical condition as an out of memory, + * the benefits of queue merging may be little relevant, or even negligible. + * + * Weight-raised queues can be merged only if their weight-raising + * period has just started. In fact cooperating processes are usually + * started together. Thus, with this filter we avoid false positives + * that would jeopardize low-latency guarantees. + * + * WARNING: queue merging may impair fairness among non-weight raised + * queues, for at least two reasons: 1) the original weight of a + * merged queue may change during the merged state, 2) even being the + * weight the same, a merged queue may be bloated with many more + * requests than the ones produced by its originally-associated + * process. + */ +static struct bfq_queue * +bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, + void *io_struct, bool request) +{ + struct bfq_queue *in_service_bfqq, *new_bfqq; + + if (bfqq->new_bfqq) + return bfqq->new_bfqq; + + if (io_struct && wr_from_too_long(bfqq) && + likely(bfqq != &bfqd->oom_bfqq)) + bfq_log_bfqq(bfqd, bfqq, + "would have looked for coop, but bfq%d wr", + bfqq->pid); + + if (!io_struct || + wr_from_too_long(bfqq) || + unlikely(bfqq == &bfqd->oom_bfqq)) + return NULL; + + /* If there is only one backlogged queue, don't search. */ + if (bfqd->busy_queues == 1) + return NULL; + + in_service_bfqq = bfqd->in_service_queue; + + if (in_service_bfqq && in_service_bfqq != bfqq && + wr_from_too_long(in_service_bfqq) + && likely(in_service_bfqq == &bfqd->oom_bfqq)) + bfq_log_bfqq(bfqd, bfqq, + "would have tried merge with in-service-queue, but wr"); + + if (!in_service_bfqq || in_service_bfqq == bfqq + || wr_from_too_long(in_service_bfqq) || + unlikely(in_service_bfqq == &bfqd->oom_bfqq)) + goto check_scheduled; + + if (bfq_rq_close_to_sector(io_struct, request, bfqd->last_position) && + bfqq->entity.parent == in_service_bfqq->entity.parent && + bfq_may_be_close_cooperator(bfqq, in_service_bfqq)) { + new_bfqq = bfq_setup_merge(bfqq, in_service_bfqq); + if (new_bfqq) + return new_bfqq; + } + /* + * Check whether there is a cooperator among currently scheduled + * queues. The only thing we need is that the bio/request is not + * NULL, as we need it to establish whether a cooperator exists. + */ +check_scheduled: + new_bfqq = bfq_find_close_cooperator(bfqd, bfqq, + bfq_io_struct_pos(io_struct, request)); + + BUG_ON(new_bfqq && bfqq->entity.parent != new_bfqq->entity.parent); + + if (new_bfqq && wr_from_too_long(new_bfqq) && + likely(new_bfqq != &bfqd->oom_bfqq) && + bfq_may_be_close_cooperator(bfqq, new_bfqq)) + bfq_log_bfqq(bfqd, bfqq, + "would have merged with bfq%d, but wr", + new_bfqq->pid); + + if (new_bfqq && !wr_from_too_long(new_bfqq) && + likely(new_bfqq != &bfqd->oom_bfqq) && + bfq_may_be_close_cooperator(bfqq, new_bfqq)) + return bfq_setup_merge(bfqq, new_bfqq); + + return NULL; +} + +static void bfq_bfqq_save_state(struct bfq_queue *bfqq) +{ + struct bfq_io_cq *bic = bfqq->bic; + + /* + * If !bfqq->bic, the queue is already shared or its requests + * have already been redirected to a shared queue; both idle window + * and weight raising state have already been saved. Do nothing. + */ + if (!bic) + return; + + bic->saved_ttime = bfqq->ttime; + bic->saved_has_short_ttime = bfq_bfqq_has_short_ttime(bfqq); + bic->saved_IO_bound = bfq_bfqq_IO_bound(bfqq); + bic->saved_in_large_burst = bfq_bfqq_in_large_burst(bfqq); + bic->was_in_burst_list = !hlist_unhashed(&bfqq->burst_list_node); + bic->saved_wr_coeff = bfqq->wr_coeff; + bic->saved_wr_start_at_switch_to_srt = bfqq->wr_start_at_switch_to_srt; + bic->saved_last_wr_start_finish = bfqq->last_wr_start_finish; + bic->saved_wr_cur_max_time = bfqq->wr_cur_max_time; + BUG_ON(time_is_after_jiffies(bfqq->last_wr_start_finish)); +} + +static void +bfq_merge_bfqqs(struct bfq_data *bfqd, struct bfq_io_cq *bic, + struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) +{ + bfq_log_bfqq(bfqd, bfqq, "merging with queue %lu", + (unsigned long) new_bfqq->pid); + BUG_ON(bfqq->bic && bfqq->bic == new_bfqq->bic); + /* Save weight raising and idle window of the merged queues */ + bfq_bfqq_save_state(bfqq); + bfq_bfqq_save_state(new_bfqq); + + if (bfq_bfqq_IO_bound(bfqq)) + bfq_mark_bfqq_IO_bound(new_bfqq); + bfq_clear_bfqq_IO_bound(bfqq); + + /* + * If bfqq is weight-raised, then let new_bfqq inherit + * weight-raising. To reduce false positives, neglect the case + * where bfqq has just been created, but has not yet made it + * to be weight-raised (which may happen because EQM may merge + * bfqq even before bfq_add_request is executed for the first + * time for bfqq). Handling this case would however be very + * easy, thanks to the flag just_created. + */ + if (new_bfqq->wr_coeff == 1 && bfqq->wr_coeff > 1) { + new_bfqq->wr_coeff = bfqq->wr_coeff; + new_bfqq->wr_cur_max_time = bfqq->wr_cur_max_time; + new_bfqq->last_wr_start_finish = bfqq->last_wr_start_finish; + new_bfqq->wr_start_at_switch_to_srt = + bfqq->wr_start_at_switch_to_srt; + if (bfq_bfqq_busy(new_bfqq)) { + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + } + + new_bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqd, new_bfqq, + "wr start after merge with %d, rais_max_time %u", + bfqq->pid, + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } + + if (bfqq->wr_coeff > 1) { /* bfqq has given its wr to new_bfqq */ + bfqq->wr_coeff = 1; + bfqq->entity.prio_changed = 1; + if (bfq_bfqq_busy(bfqq)) { + bfqd->wr_busy_queues--; + BUG_ON(bfqd->wr_busy_queues < 0); + } + + } + + bfq_log_bfqq(bfqd, new_bfqq, "merge_bfqqs: wr_busy %d", + bfqd->wr_busy_queues); + + /* + * Merge queues (that is, let bic redirect its requests to new_bfqq) + */ + bic_set_bfqq(bic, new_bfqq, 1); + bfq_mark_bfqq_coop(new_bfqq); + /* + * new_bfqq now belongs to at least two bics (it is a shared queue): + * set new_bfqq->bic to NULL. bfqq either: + * - does not belong to any bic any more, and hence bfqq->bic must + * be set to NULL, or + * - is a queue whose owning bics have already been redirected to a + * different queue, hence the queue is destined to not belong to + * any bic soon and bfqq->bic is already NULL (therefore the next + * assignment causes no harm). + */ + new_bfqq->bic = NULL; + bfqq->bic = NULL; + /* release process reference to bfqq */ + bfq_put_queue(bfqq); +} + +static bool bfq_allow_bio_merge(struct request_queue *q, struct request *rq, + struct bio *bio) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + bool is_sync = op_is_sync(bio->bi_opf); + struct bfq_queue *bfqq = bfqd->bio_bfqq, *new_bfqq; + + assert_spin_locked(&bfqd->lock); + /* + * Disallow merge of a sync bio into an async request. + */ + if (is_sync && !rq_is_sync(rq)) + return false; + + /* + * Lookup the bfqq that this bio will be queued with. Allow + * merge only if rq is queued there. + */ + BUG_ON(!bfqd->bio_bfqq_set); + if (!bfqq) + return false; + + /* + * We take advantage of this function to perform an early merge + * of the queues of possible cooperating processes. + */ + new_bfqq = bfq_setup_cooperator(bfqd, bfqq, bio, false); + BUG_ON(new_bfqq == bfqq); + if (new_bfqq) { + /* + * bic still points to bfqq, then it has not yet been + * redirected to some other bfq_queue, and a queue + * merge beween bfqq and new_bfqq can be safely + * fulfillled, i.e., bic can be redirected to new_bfqq + * and bfqq can be put. + */ + bfq_merge_bfqqs(bfqd, bfqd->bio_bic, bfqq, + new_bfqq); + /* + * If we get here, bio will be queued into new_queue, + * so use new_bfqq to decide whether bio and rq can be + * merged. + */ + bfqq = new_bfqq; + + /* + * Change also bqfd->bio_bfqq, as + * bfqd->bio_bic now points to new_bfqq, and + * this function may be invoked again (and then may + * use again bqfd->bio_bfqq). + */ + bfqd->bio_bfqq = bfqq; + } + return bfqq == RQ_BFQQ(rq); +} + +/* + * Set the maximum time for the in-service queue to consume its + * budget. This prevents seeky processes from lowering the throughput. + * In practice, a time-slice service scheme is used with seeky + * processes. + */ +static void bfq_set_budget_timeout(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + unsigned int timeout_coeff; + + if (bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time) + timeout_coeff = 1; + else + timeout_coeff = bfqq->entity.weight / bfqq->entity.orig_weight; + + bfqd->last_budget_start = ktime_get(); + + bfqq->budget_timeout = jiffies + + bfqd->bfq_timeout * timeout_coeff; + + bfq_log_bfqq(bfqd, bfqq, "set budget_timeout %u", + jiffies_to_msecs(bfqd->bfq_timeout * timeout_coeff)); +} + +static void __bfq_set_in_service_queue(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + if (bfqq) { + bfqg_stats_update_avg_queue_size(bfqq_group(bfqq)); + bfq_clear_bfqq_fifo_expire(bfqq); + + bfqd->budgets_assigned = (bfqd->budgets_assigned*7 + 256) / 8; + + BUG_ON(bfqq == bfqd->in_service_queue); + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + + if (time_is_before_jiffies(bfqq->last_wr_start_finish) && + bfqq->wr_coeff > 1 && + bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time && + time_is_before_jiffies(bfqq->budget_timeout)) { + /* + * For soft real-time queues, move the start + * of the weight-raising period forward by the + * time the queue has not received any + * service. Otherwise, a relatively long + * service delay is likely to cause the + * weight-raising period of the queue to end, + * because of the short duration of the + * weight-raising period of a soft real-time + * queue. It is worth noting that this move + * is not so dangerous for the other queues, + * because soft real-time queues are not + * greedy. + * + * To not add a further variable, we use the + * overloaded field budget_timeout to + * determine for how long the queue has not + * received service, i.e., how much time has + * elapsed since the queue expired. However, + * this is a little imprecise, because + * budget_timeout is set to jiffies if bfqq + * not only expires, but also remains with no + * request. + */ + if (time_after(bfqq->budget_timeout, + bfqq->last_wr_start_finish)) + bfqq->last_wr_start_finish += + jiffies - bfqq->budget_timeout; + else + bfqq->last_wr_start_finish = jiffies; + + if (time_is_after_jiffies(bfqq->last_wr_start_finish)) { + pr_crit( + "BFQ WARNING:last %lu budget %lu jiffies %lu", + bfqq->last_wr_start_finish, + bfqq->budget_timeout, + jiffies); + pr_crit("diff %lu", jiffies - + max_t(unsigned long, + bfqq->last_wr_start_finish, + bfqq->budget_timeout)); + bfqq->last_wr_start_finish = jiffies; + } + } + + bfq_set_budget_timeout(bfqd, bfqq); + bfq_log_bfqq(bfqd, bfqq, + "set_in_service_queue, cur-budget = %d", + bfqq->entity.budget); + } else + bfq_log(bfqd, "set_in_service_queue: NULL"); + + bfqd->in_service_queue = bfqq; +} + +/* + * Get and set a new queue for service. + */ +static struct bfq_queue *bfq_set_in_service_queue(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq = bfq_get_next_queue(bfqd); + + __bfq_set_in_service_queue(bfqd, bfqq); + return bfqq; +} + +static void bfq_arm_slice_timer(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq = bfqd->in_service_queue; + u32 sl; + + BUG_ON(!RB_EMPTY_ROOT(&bfqq->sort_list)); + + bfq_mark_bfqq_wait_request(bfqq); + + /* + * We don't want to idle for seeks, but we do want to allow + * fair distribution of slice time for a process doing back-to-back + * seeks. So allow a little bit of time for him to submit a new rq. + * + * To prevent processes with (partly) seeky workloads from + * being too ill-treated, grant them a small fraction of the + * assigned budget before reducing the waiting time to + * BFQ_MIN_TT. This happened to help reduce latency. + */ + sl = bfqd->bfq_slice_idle; + /* + * Unless the queue is being weight-raised or the scenario is + * asymmetric, grant only minimum idle time if the queue + * is seeky. A long idling is preserved for a weight-raised + * queue, or, more in general, in an asymemtric scenario, + * because a long idling is needed for guaranteeing to a queue + * its reserved share of the throughput (in particular, it is + * needed if the queue has a higher weight than some other + * queue). + */ + if (BFQQ_SEEKY(bfqq) && bfqq->wr_coeff == 1 && + bfq_symmetric_scenario(bfqd)) + sl = min_t(u32, sl, BFQ_MIN_TT); + + bfqd->last_idling_start = ktime_get(); + hrtimer_start(&bfqd->idle_slice_timer, ns_to_ktime(sl), + HRTIMER_MODE_REL); + bfqg_stats_set_start_idle_time(bfqq_group(bfqq)); + bfq_log(bfqd, "arm idle: %ld/%ld ms", + sl / NSEC_PER_MSEC, bfqd->bfq_slice_idle / NSEC_PER_MSEC); +} + +/* + * In autotuning mode, max_budget is dynamically recomputed as the + * amount of sectors transferred in timeout at the estimated peak + * rate. This enables BFQ to utilize a full timeslice with a full + * budget, even if the in-service queue is served at peak rate. And + * this maximises throughput with sequential workloads. + */ +static unsigned long bfq_calc_max_budget(struct bfq_data *bfqd) +{ + return (u64)bfqd->peak_rate * USEC_PER_MSEC * + jiffies_to_msecs(bfqd->bfq_timeout)>>BFQ_RATE_SHIFT; +} + +/* + * Update parameters related to throughput and responsiveness, as a + * function of the estimated peak rate. See comments on + * bfq_calc_max_budget(), and on T_slow and T_fast arrays. + */ +static void update_thr_responsiveness_params(struct bfq_data *bfqd) +{ + int dev_type = blk_queue_nonrot(bfqd->queue); + + if (bfqd->bfq_user_max_budget == 0) { + bfqd->bfq_max_budget = + bfq_calc_max_budget(bfqd); + BUG_ON(bfqd->bfq_max_budget < 0); + bfq_log(bfqd, "new max_budget = %d", + bfqd->bfq_max_budget); + } + + if (bfqd->device_speed == BFQ_BFQD_FAST && + bfqd->peak_rate < device_speed_thresh[dev_type]) { + bfqd->device_speed = BFQ_BFQD_SLOW; + bfqd->RT_prod = R_slow[dev_type] * + T_slow[dev_type]; + } else if (bfqd->device_speed == BFQ_BFQD_SLOW && + bfqd->peak_rate > device_speed_thresh[dev_type]) { + bfqd->device_speed = BFQ_BFQD_FAST; + bfqd->RT_prod = R_fast[dev_type] * + T_fast[dev_type]; + } + + bfq_log(bfqd, +"dev_type %s dev_speed_class = %s (%llu sects/sec), thresh %llu setcs/sec", + dev_type == 0 ? "ROT" : "NONROT", + bfqd->device_speed == BFQ_BFQD_FAST ? "FAST" : "SLOW", + bfqd->device_speed == BFQ_BFQD_FAST ? + (USEC_PER_SEC*(u64)R_fast[dev_type])>>BFQ_RATE_SHIFT : + (USEC_PER_SEC*(u64)R_slow[dev_type])>>BFQ_RATE_SHIFT, + (USEC_PER_SEC*(u64)device_speed_thresh[dev_type])>> + BFQ_RATE_SHIFT); +} + +static void bfq_reset_rate_computation(struct bfq_data *bfqd, struct request *rq) +{ + if (rq != NULL) { /* new rq dispatch now, reset accordingly */ + bfqd->last_dispatch = bfqd->first_dispatch = ktime_get_ns() ; + bfqd->peak_rate_samples = 1; + bfqd->sequential_samples = 0; + bfqd->tot_sectors_dispatched = bfqd->last_rq_max_size = + blk_rq_sectors(rq); + } else /* no new rq dispatched, just reset the number of samples */ + bfqd->peak_rate_samples = 0; /* full re-init on next disp. */ + + bfq_log(bfqd, + "reset_rate_computation at end, sample %u/%u tot_sects %llu", + bfqd->peak_rate_samples, bfqd->sequential_samples, + bfqd->tot_sectors_dispatched); +} + +static void bfq_update_rate_reset(struct bfq_data *bfqd, struct request *rq) +{ + u32 rate, weight, divisor; + + /* + * For the convergence property to hold (see comments on + * bfq_update_peak_rate()) and for the assessment to be + * reliable, a minimum number of samples must be present, and + * a minimum amount of time must have elapsed. If not so, do + * not compute new rate. Just reset parameters, to get ready + * for a new evaluation attempt. + */ + if (bfqd->peak_rate_samples < BFQ_RATE_MIN_SAMPLES || + bfqd->delta_from_first < BFQ_RATE_MIN_INTERVAL) { + bfq_log(bfqd, + "update_rate_reset: only resetting, delta_first %lluus samples %d", + bfqd->delta_from_first>>10, bfqd->peak_rate_samples); + goto reset_computation; + } + + /* + * If a new request completion has occurred after last + * dispatch, then, to approximate the rate at which requests + * have been served by the device, it is more precise to + * extend the observation interval to the last completion. + */ + bfqd->delta_from_first = + max_t(u64, bfqd->delta_from_first, + bfqd->last_completion - bfqd->first_dispatch); + + BUG_ON(bfqd->delta_from_first == 0); + /* + * Rate computed in sects/usec, and not sects/nsec, for + * precision issues. + */ + rate = div64_ul(bfqd->tot_sectors_dispatched<delta_from_first, NSEC_PER_USEC)); + + bfq_log(bfqd, +"update_rate_reset: tot_sects %llu delta_first %lluus rate %llu sects/s (%d)", + bfqd->tot_sectors_dispatched, bfqd->delta_from_first>>10, + ((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT), + rate > 20< 20M sectors/sec) + */ + if ((bfqd->sequential_samples < (3 * bfqd->peak_rate_samples)>>2 && + rate <= bfqd->peak_rate) || + rate > 20<peak_rate_samples, bfqd->sequential_samples, + ((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT), + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT)); + goto reset_computation; + } else { + bfq_log(bfqd, + "update_rate_reset: do update, samples %u/%u rate/peak %llu/%llu", + bfqd->peak_rate_samples, bfqd->sequential_samples, + ((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT), + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT)); + } + + /* + * We have to update the peak rate, at last! To this purpose, + * we use a low-pass filter. We compute the smoothing constant + * of the filter as a function of the 'weight' of the new + * measured rate. + * + * As can be seen in next formulas, we define this weight as a + * quantity proportional to how sequential the workload is, + * and to how long the observation time interval is. + * + * The weight runs from 0 to 8. The maximum value of the + * weight, 8, yields the minimum value for the smoothing + * constant. At this minimum value for the smoothing constant, + * the measured rate contributes for half of the next value of + * the estimated peak rate. + * + * So, the first step is to compute the weight as a function + * of how sequential the workload is. Note that the weight + * cannot reach 9, because bfqd->sequential_samples cannot + * become equal to bfqd->peak_rate_samples, which, in its + * turn, holds true because bfqd->sequential_samples is not + * incremented for the first sample. + */ + weight = (9 * bfqd->sequential_samples) / bfqd->peak_rate_samples; + + /* + * Second step: further refine the weight as a function of the + * duration of the observation interval. + */ + weight = min_t(u32, 8, + div_u64(weight * bfqd->delta_from_first, + BFQ_RATE_REF_INTERVAL)); + + /* + * Divisor ranging from 10, for minimum weight, to 2, for + * maximum weight. + */ + divisor = 10 - weight; + BUG_ON(divisor == 0); + + /* + * Finally, update peak rate: + * + * peak_rate = peak_rate * (divisor-1) / divisor + rate / divisor + */ + bfqd->peak_rate *= divisor-1; + bfqd->peak_rate /= divisor; + rate /= divisor; /* smoothing constant alpha = 1/divisor */ + + bfq_log(bfqd, + "update_rate_reset: divisor %d tmp_peak_rate %llu tmp_rate %u", + divisor, + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT), + (u32)((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT)); + + BUG_ON(bfqd->peak_rate == 0); + BUG_ON(bfqd->peak_rate > 20<peak_rate += rate; + update_thr_responsiveness_params(bfqd); + BUG_ON(bfqd->peak_rate > 20<peak_rate_samples == 0) { /* first dispatch */ + bfq_log(bfqd, + "update_peak_rate: goto reset, samples %d", + bfqd->peak_rate_samples) ; + bfq_reset_rate_computation(bfqd, rq); + goto update_last_values; /* will add one sample */ + } + + /* + * Device idle for very long: the observation interval lasting + * up to this dispatch cannot be a valid observation interval + * for computing a new peak rate (similarly to the late- + * completion event in bfq_completed_request()). Go to + * update_rate_and_reset to have the following three steps + * taken: + * - close the observation interval at the last (previous) + * request dispatch or completion + * - compute rate, if possible, for that observation interval + * - start a new observation interval with this dispatch + */ + if (now_ns - bfqd->last_dispatch > 100*NSEC_PER_MSEC && + bfqd->rq_in_driver == 0) { + bfq_log(bfqd, +"update_peak_rate: jumping to updating&resetting delta_last %lluus samples %d", + (now_ns - bfqd->last_dispatch)>>10, + bfqd->peak_rate_samples) ; + goto update_rate_and_reset; + } + + /* Update sampling information */ + bfqd->peak_rate_samples++; + + if ((bfqd->rq_in_driver > 0 || + now_ns - bfqd->last_completion < BFQ_MIN_TT) + && get_sdist(bfqd->last_position, rq) < BFQQ_SEEK_THR) + bfqd->sequential_samples++; + + bfqd->tot_sectors_dispatched += blk_rq_sectors(rq); + + /* Reset max observed rq size every 32 dispatches */ + if (likely(bfqd->peak_rate_samples % 32)) + bfqd->last_rq_max_size = + max_t(u32, blk_rq_sectors(rq), bfqd->last_rq_max_size); + else + bfqd->last_rq_max_size = blk_rq_sectors(rq); + + bfqd->delta_from_first = now_ns - bfqd->first_dispatch; + + bfq_log(bfqd, + "update_peak_rate: added samples %u/%u tot_sects %llu delta_first %lluus", + bfqd->peak_rate_samples, bfqd->sequential_samples, + bfqd->tot_sectors_dispatched, + bfqd->delta_from_first>>10); + + /* Target observation interval not yet reached, go on sampling */ + if (bfqd->delta_from_first < BFQ_RATE_REF_INTERVAL) + goto update_last_values; + +update_rate_and_reset: + bfq_update_rate_reset(bfqd, rq); +update_last_values: + bfqd->last_position = blk_rq_pos(rq) + blk_rq_sectors(rq); + bfqd->last_dispatch = now_ns; + + bfq_log(bfqd, + "update_peak_rate: delta_first %lluus last_pos %llu peak_rate %llu", + (now_ns - bfqd->first_dispatch)>>10, + (unsigned long long) bfqd->last_position, + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT)); + bfq_log(bfqd, + "update_peak_rate: samples at end %d", bfqd->peak_rate_samples); +} + +/* + * Remove request from internal lists. + */ +static void bfq_dispatch_remove(struct request_queue *q, struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + + /* + * For consistency, the next instruction should have been + * executed after removing the request from the queue and + * dispatching it. We execute instead this instruction before + * bfq_remove_request() (and hence introduce a temporary + * inconsistency), for efficiency. In fact, should this + * dispatch occur for a non in-service bfqq, this anticipated + * increment prevents two counters related to bfqq->dispatched + * from risking to be, first, uselessly decremented, and then + * incremented again when the (new) value of bfqq->dispatched + * happens to be taken into account. + */ + bfqq->dispatched++; + bfq_update_peak_rate(q->elevator->elevator_data, rq); + + bfq_remove_request(q, rq); +} + +static void __bfq_bfqq_expire(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + BUG_ON(bfqq != bfqd->in_service_queue); + + /* + * If this bfqq is shared between multiple processes, check + * to make sure that those processes are still issuing I/Os + * within the mean seek distance. If not, it may be time to + * break the queues apart again. + */ + if (bfq_bfqq_coop(bfqq) && BFQQ_SEEKY(bfqq)) + bfq_mark_bfqq_split_coop(bfqq); + + if (RB_EMPTY_ROOT(&bfqq->sort_list)) { + if (bfqq->dispatched == 0) + /* + * Overloading budget_timeout field to store + * the time at which the queue remains with no + * backlog and no outstanding request; used by + * the weight-raising mechanism. + */ + bfqq->budget_timeout = jiffies; + + bfq_del_bfqq_busy(bfqd, bfqq, true); + } else { + bfq_requeue_bfqq(bfqd, bfqq, true); + /* + * Resort priority tree of potential close cooperators. + */ + bfq_pos_tree_add_move(bfqd, bfqq); + } + + /* + * All in-service entities must have been properly deactivated + * or requeued before executing the next function, which + * resets all in-service entites as no more in service. + */ + __bfq_bfqd_reset_in_service(bfqd); +} + +/** + * __bfq_bfqq_recalc_budget - try to adapt the budget to the @bfqq behavior. + * @bfqd: device data. + * @bfqq: queue to update. + * @reason: reason for expiration. + * + * Handle the feedback on @bfqq budget at queue expiration. + * See the body for detailed comments. + */ +static void __bfq_bfqq_recalc_budget(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + enum bfqq_expiration reason) +{ + struct request *next_rq; + int budget, min_budget; + + BUG_ON(bfqq != bfqd->in_service_queue); + + min_budget = bfq_min_budget(bfqd); + + if (bfqq->wr_coeff == 1) + budget = bfqq->max_budget; + else /* + * Use a constant, low budget for weight-raised queues, + * to help achieve a low latency. Keep it slightly higher + * than the minimum possible budget, to cause a little + * bit fewer expirations. + */ + budget = 2 * min_budget; + + bfq_log_bfqq(bfqd, bfqq, "recalc_budg: last budg %d, budg left %d", + bfqq->entity.budget, bfq_bfqq_budget_left(bfqq)); + bfq_log_bfqq(bfqd, bfqq, "recalc_budg: last max_budg %d, min budg %d", + budget, bfq_min_budget(bfqd)); + bfq_log_bfqq(bfqd, bfqq, "recalc_budg: sync %d, seeky %d", + bfq_bfqq_sync(bfqq), BFQQ_SEEKY(bfqd->in_service_queue)); + + if (bfq_bfqq_sync(bfqq) && bfqq->wr_coeff == 1) { + switch (reason) { + /* + * Caveat: in all the following cases we trade latency + * for throughput. + */ + case BFQ_BFQQ_TOO_IDLE: + /* + * This is the only case where we may reduce + * the budget: if there is no request of the + * process still waiting for completion, then + * we assume (tentatively) that the timer has + * expired because the batch of requests of + * the process could have been served with a + * smaller budget. Hence, betting that + * process will behave in the same way when it + * becomes backlogged again, we reduce its + * next budget. As long as we guess right, + * this budget cut reduces the latency + * experienced by the process. + * + * However, if there are still outstanding + * requests, then the process may have not yet + * issued its next request just because it is + * still waiting for the completion of some of + * the still outstanding ones. So in this + * subcase we do not reduce its budget, on the + * contrary we increase it to possibly boost + * the throughput, as discussed in the + * comments to the BUDGET_TIMEOUT case. + */ + if (bfqq->dispatched > 0) /* still outstanding reqs */ + budget = min(budget * 2, bfqd->bfq_max_budget); + else { + if (budget > 5 * min_budget) + budget -= 4 * min_budget; + else + budget = min_budget; + } + break; + case BFQ_BFQQ_BUDGET_TIMEOUT: + /* + * We double the budget here because it gives + * the chance to boost the throughput if this + * is not a seeky process (and has bumped into + * this timeout because of, e.g., ZBR). + */ + budget = min(budget * 2, bfqd->bfq_max_budget); + break; + case BFQ_BFQQ_BUDGET_EXHAUSTED: + /* + * The process still has backlog, and did not + * let either the budget timeout or the disk + * idling timeout expire. Hence it is not + * seeky, has a short thinktime and may be + * happy with a higher budget too. So + * definitely increase the budget of this good + * candidate to boost the disk throughput. + */ + budget = min(budget * 4, bfqd->bfq_max_budget); + break; + case BFQ_BFQQ_NO_MORE_REQUESTS: + /* + * For queues that expire for this reason, it + * is particularly important to keep the + * budget close to the actual service they + * need. Doing so reduces the timestamp + * misalignment problem described in the + * comments in the body of + * __bfq_activate_entity. In fact, suppose + * that a queue systematically expires for + * BFQ_BFQQ_NO_MORE_REQUESTS and presents a + * new request in time to enjoy timestamp + * back-shifting. The larger the budget of the + * queue is with respect to the service the + * queue actually requests in each service + * slot, the more times the queue can be + * reactivated with the same virtual finish + * time. It follows that, even if this finish + * time is pushed to the system virtual time + * to reduce the consequent timestamp + * misalignment, the queue unjustly enjoys for + * many re-activations a lower finish time + * than all newly activated queues. + * + * The service needed by bfqq is measured + * quite precisely by bfqq->entity.service. + * Since bfqq does not enjoy device idling, + * bfqq->entity.service is equal to the number + * of sectors that the process associated with + * bfqq requested to read/write before waiting + * for request completions, or blocking for + * other reasons. + */ + budget = max_t(int, bfqq->entity.service, min_budget); + break; + default: + return; + } + } else if (!bfq_bfqq_sync(bfqq)) + /* + * Async queues get always the maximum possible + * budget, as for them we do not care about latency + * (in addition, their ability to dispatch is limited + * by the charging factor). + */ + budget = bfqd->bfq_max_budget; + + bfqq->max_budget = budget; + + if (bfqd->budgets_assigned >= bfq_stats_min_budgets && + !bfqd->bfq_user_max_budget) + bfqq->max_budget = min(bfqq->max_budget, bfqd->bfq_max_budget); + + /* + * If there is still backlog, then assign a new budget, making + * sure that it is large enough for the next request. Since + * the finish time of bfqq must be kept in sync with the + * budget, be sure to call __bfq_bfqq_expire() *after* this + * update. + * + * If there is no backlog, then no need to update the budget; + * it will be updated on the arrival of a new request. + */ + next_rq = bfqq->next_rq; + if (next_rq) { + BUG_ON(reason == BFQ_BFQQ_TOO_IDLE || + reason == BFQ_BFQQ_NO_MORE_REQUESTS); + bfqq->entity.budget = max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(next_rq, bfqq)); + BUG_ON(!bfq_bfqq_busy(bfqq)); + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + } + + bfq_log_bfqq(bfqd, bfqq, "head sect: %u, new budget %d", + next_rq ? blk_rq_sectors(next_rq) : 0, + bfqq->entity.budget); +} + +/* + * Return true if the process associated with bfqq is "slow". The slow + * flag is used, in addition to the budget timeout, to reduce the + * amount of service provided to seeky processes, and thus reduce + * their chances to lower the throughput. More details in the comments + * on the function bfq_bfqq_expire(). + * + * An important observation is in order: as discussed in the comments + * on the function bfq_update_peak_rate(), with devices with internal + * queues, it is hard if ever possible to know when and for how long + * an I/O request is processed by the device (apart from the trivial + * I/O pattern where a new request is dispatched only after the + * previous one has been completed). This makes it hard to evaluate + * the real rate at which the I/O requests of each bfq_queue are + * served. In fact, for an I/O scheduler like BFQ, serving a + * bfq_queue means just dispatching its requests during its service + * slot (i.e., until the budget of the queue is exhausted, or the + * queue remains idle, or, finally, a timeout fires). But, during the + * service slot of a bfq_queue, around 100 ms at most, the device may + * be even still processing requests of bfq_queues served in previous + * service slots. On the opposite end, the requests of the in-service + * bfq_queue may be completed after the service slot of the queue + * finishes. + * + * Anyway, unless more sophisticated solutions are used + * (where possible), the sum of the sizes of the requests dispatched + * during the service slot of a bfq_queue is probably the only + * approximation available for the service received by the bfq_queue + * during its service slot. And this sum is the quantity used in this + * function to evaluate the I/O speed of a process. + */ +static bool bfq_bfqq_is_slow(struct bfq_data *bfqd, struct bfq_queue *bfqq, + bool compensate, enum bfqq_expiration reason, + unsigned long *delta_ms) +{ + ktime_t delta_ktime; + u32 delta_usecs; + bool slow = BFQQ_SEEKY(bfqq); /* if delta too short, use seekyness */ + + if (!bfq_bfqq_sync(bfqq)) + return false; + + if (compensate) + delta_ktime = bfqd->last_idling_start; + else + delta_ktime = ktime_get(); + delta_ktime = ktime_sub(delta_ktime, bfqd->last_budget_start); + delta_usecs = ktime_to_us(delta_ktime); + + /* don't use too short time intervals */ + if (delta_usecs < 1000) { + if (blk_queue_nonrot(bfqd->queue)) + /* + * give same worst-case guarantees as idling + * for seeky + */ + *delta_ms = BFQ_MIN_TT / NSEC_PER_MSEC; + else /* charge at least one seek */ + *delta_ms = bfq_slice_idle / NSEC_PER_MSEC; + + bfq_log(bfqd, "bfq_bfqq_is_slow: too short %u", delta_usecs); + + return slow; + } + + *delta_ms = delta_usecs / USEC_PER_MSEC; + + /* + * Use only long (> 20ms) intervals to filter out excessive + * spikes in service rate estimation. + */ + if (delta_usecs > 20000) { + /* + * Caveat for rotational devices: processes doing I/O + * in the slower disk zones tend to be slow(er) even + * if not seeky. In this respect, the estimated peak + * rate is likely to be an average over the disk + * surface. Accordingly, to not be too harsh with + * unlucky processes, a process is deemed slow only if + * its rate has been lower than half of the estimated + * peak rate. + */ + slow = bfqq->entity.service < bfqd->bfq_max_budget / 2; + bfq_log(bfqd, "bfq_bfqq_is_slow: relative rate %d/%d", + bfqq->entity.service, bfqd->bfq_max_budget); + } + + bfq_log_bfqq(bfqd, bfqq, "bfq_bfqq_is_slow: slow %d", slow); + + return slow; +} + +/* + * To be deemed as soft real-time, an application must meet two + * requirements. First, the application must not require an average + * bandwidth higher than the approximate bandwidth required to playback or + * record a compressed high-definition video. + * The next function is invoked on the completion of the last request of a + * batch, to compute the next-start time instant, soft_rt_next_start, such + * that, if the next request of the application does not arrive before + * soft_rt_next_start, then the above requirement on the bandwidth is met. + * + * The second requirement is that the request pattern of the application is + * isochronous, i.e., that, after issuing a request or a batch of requests, + * the application stops issuing new requests until all its pending requests + * have been completed. After that, the application may issue a new batch, + * and so on. + * For this reason the next function is invoked to compute + * soft_rt_next_start only for applications that meet this requirement, + * whereas soft_rt_next_start is set to infinity for applications that do + * not. + * + * Unfortunately, even a greedy application may happen to behave in an + * isochronous way if the CPU load is high. In fact, the application may + * stop issuing requests while the CPUs are busy serving other processes, + * then restart, then stop again for a while, and so on. In addition, if + * the disk achieves a low enough throughput with the request pattern + * issued by the application (e.g., because the request pattern is random + * and/or the device is slow), then the application may meet the above + * bandwidth requirement too. To prevent such a greedy application to be + * deemed as soft real-time, a further rule is used in the computation of + * soft_rt_next_start: soft_rt_next_start must be higher than the current + * time plus the maximum time for which the arrival of a request is waited + * for when a sync queue becomes idle, namely bfqd->bfq_slice_idle. + * This filters out greedy applications, as the latter issue instead their + * next request as soon as possible after the last one has been completed + * (in contrast, when a batch of requests is completed, a soft real-time + * application spends some time processing data). + * + * Unfortunately, the last filter may easily generate false positives if + * only bfqd->bfq_slice_idle is used as a reference time interval and one + * or both the following cases occur: + * 1) HZ is so low that the duration of a jiffy is comparable to or higher + * than bfqd->bfq_slice_idle. This happens, e.g., on slow devices with + * HZ=100. + * 2) jiffies, instead of increasing at a constant rate, may stop increasing + * for a while, then suddenly 'jump' by several units to recover the lost + * increments. This seems to happen, e.g., inside virtual machines. + * To address this issue, we do not use as a reference time interval just + * bfqd->bfq_slice_idle, but bfqd->bfq_slice_idle plus a few jiffies. In + * particular we add the minimum number of jiffies for which the filter + * seems to be quite precise also in embedded systems and KVM/QEMU virtual + * machines. + */ +static unsigned long bfq_bfqq_softrt_next_start(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqd, bfqq, +"softrt_next_start: service_blkg %lu soft_rate %u sects/sec interval %u", + bfqq->service_from_backlogged, + bfqd->bfq_wr_max_softrt_rate, + jiffies_to_msecs(HZ * bfqq->service_from_backlogged / + bfqd->bfq_wr_max_softrt_rate)); + + return max(bfqq->last_idle_bklogged + + HZ * bfqq->service_from_backlogged / + bfqd->bfq_wr_max_softrt_rate, + jiffies + nsecs_to_jiffies(bfqq->bfqd->bfq_slice_idle) + 4); +} + +/** + * bfq_bfqq_expire - expire a queue. + * @bfqd: device owning the queue. + * @bfqq: the queue to expire. + * @compensate: if true, compensate for the time spent idling. + * @reason: the reason causing the expiration. + * + * If the process associated with bfqq does slow I/O (e.g., because it + * issues random requests), we charge bfqq with the time it has been + * in service instead of the service it has received (see + * bfq_bfqq_charge_time for details on how this goal is achieved). As + * a consequence, bfqq will typically get higher timestamps upon + * reactivation, and hence it will be rescheduled as if it had + * received more service than what it has actually received. In the + * end, bfqq receives less service in proportion to how slowly its + * associated process consumes its budgets (and hence how seriously it + * tends to lower the throughput). In addition, this time-charging + * strategy guarantees time fairness among slow processes. In + * contrast, if the process associated with bfqq is not slow, we + * charge bfqq exactly with the service it has received. + * + * Charging time to the first type of queues and the exact service to + * the other has the effect of using the WF2Q+ policy to schedule the + * former on a timeslice basis, without violating service domain + * guarantees among the latter. + */ +static void bfq_bfqq_expire(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool compensate, + enum bfqq_expiration reason) +{ + bool slow; + unsigned long delta = 0; + struct bfq_entity *entity = &bfqq->entity; + int ref; + + BUG_ON(bfqq != bfqd->in_service_queue); + + /* + * Check whether the process is slow (see bfq_bfqq_is_slow). + */ + slow = bfq_bfqq_is_slow(bfqd, bfqq, compensate, reason, &delta); + + /* + * Increase service_from_backlogged before next statement, + * because the possible next invocation of + * bfq_bfqq_charge_time would likely inflate + * entity->service. In contrast, service_from_backlogged must + * contain real service, to enable the soft real-time + * heuristic to correctly compute the bandwidth consumed by + * bfqq. + */ + bfqq->service_from_backlogged += entity->service; + + /* + * As above explained, charge slow (typically seeky) and + * timed-out queues with the time and not the service + * received, to favor sequential workloads. + * + * Processes doing I/O in the slower disk zones will tend to + * be slow(er) even if not seeky. Therefore, since the + * estimated peak rate is actually an average over the disk + * surface, these processes may timeout just for bad luck. To + * avoid punishing them, do not charge time to processes that + * succeeded in consuming at least 2/3 of their budget. This + * allows BFQ to preserve enough elasticity to still perform + * bandwidth, and not time, distribution with little unlucky + * or quasi-sequential processes. + */ + if (bfqq->wr_coeff == 1 && + (slow || + (reason == BFQ_BFQQ_BUDGET_TIMEOUT && + bfq_bfqq_budget_left(bfqq) >= entity->budget / 3))) + bfq_bfqq_charge_time(bfqd, bfqq, delta); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + if (reason == BFQ_BFQQ_TOO_IDLE && + entity->service <= 2 * entity->budget / 10) + bfq_clear_bfqq_IO_bound(bfqq); + + if (bfqd->low_latency && bfqq->wr_coeff == 1) + bfqq->last_wr_start_finish = jiffies; + + if (bfqd->low_latency && bfqd->bfq_wr_max_softrt_rate > 0 && + RB_EMPTY_ROOT(&bfqq->sort_list)) { + /* + * If we get here, and there are no outstanding + * requests, then the request pattern is isochronous + * (see the comments on the function + * bfq_bfqq_softrt_next_start()). Thus we can compute + * soft_rt_next_start. If, instead, the queue still + * has outstanding requests, then we have to wait for + * the completion of all the outstanding requests to + * discover whether the request pattern is actually + * isochronous. + */ + BUG_ON(bfqd->busy_queues < 1); + if (bfqq->dispatched == 0) { + bfqq->soft_rt_next_start = + bfq_bfqq_softrt_next_start(bfqd, bfqq); + bfq_log_bfqq(bfqd, bfqq, "new soft_rt_next %lu", + bfqq->soft_rt_next_start); + } else { + /* + * The application is still waiting for the + * completion of one or more requests: + * prevent it from possibly being incorrectly + * deemed as soft real-time by setting its + * soft_rt_next_start to infinity. In fact, + * without this assignment, the application + * would be incorrectly deemed as soft + * real-time if: + * 1) it issued a new request before the + * completion of all its in-flight + * requests, and + * 2) at that time, its soft_rt_next_start + * happened to be in the past. + */ + bfqq->soft_rt_next_start = + bfq_greatest_from_now(); + /* + * Schedule an update of soft_rt_next_start to when + * the task may be discovered to be isochronous. + */ + bfq_mark_bfqq_softrt_update(bfqq); + } + } + + bfq_log_bfqq(bfqd, bfqq, + "expire (%d, slow %d, num_disp %d, short_ttime %d, weight %d)", + reason, slow, bfqq->dispatched, + bfq_bfqq_has_short_ttime(bfqq), entity->weight); + + /* + * Increase, decrease or leave budget unchanged according to + * reason. + */ + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + __bfq_bfqq_recalc_budget(bfqd, bfqq, reason); + BUG_ON(bfqq->next_rq == NULL && + bfqq->entity.budget < bfqq->entity.service); + ref = bfqq->ref; + __bfq_bfqq_expire(bfqd, bfqq); + + BUG_ON(ref > 1 && + !bfq_bfqq_busy(bfqq) && reason == BFQ_BFQQ_BUDGET_EXHAUSTED && + !bfq_class_idle(bfqq)); + + /* mark bfqq as waiting a request only if a bic still points to it */ + if (ref > 1 && !bfq_bfqq_busy(bfqq) && + reason != BFQ_BFQQ_BUDGET_TIMEOUT && + reason != BFQ_BFQQ_BUDGET_EXHAUSTED) + bfq_mark_bfqq_non_blocking_wait_rq(bfqq); +} + +/* + * Budget timeout is not implemented through a dedicated timer, but + * just checked on request arrivals and completions, as well as on + * idle timer expirations. + */ +static bool bfq_bfqq_budget_timeout(struct bfq_queue *bfqq) +{ + return time_is_before_eq_jiffies(bfqq->budget_timeout); +} + +/* + * If we expire a queue that is actively waiting (i.e., with the + * device idled) for the arrival of a new request, then we may incur + * the timestamp misalignment problem described in the body of the + * function __bfq_activate_entity. Hence we return true only if this + * condition does not hold, or if the queue is slow enough to deserve + * only to be kicked off for preserving a high throughput. + */ +static bool bfq_may_expire_for_budg_timeout(struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqq->bfqd, bfqq, + "may_budget_timeout: wait_request %d left %d timeout %d", + bfq_bfqq_wait_request(bfqq), + bfq_bfqq_budget_left(bfqq) >= bfqq->entity.budget / 3, + bfq_bfqq_budget_timeout(bfqq)); + + return (!bfq_bfqq_wait_request(bfqq) || + bfq_bfqq_budget_left(bfqq) >= bfqq->entity.budget / 3) + && + bfq_bfqq_budget_timeout(bfqq); +} + +/* + * For a queue that becomes empty, device idling is allowed only if + * this function returns true for that queue. As a consequence, since + * device idling plays a critical role for both throughput boosting + * and service guarantees, the return value of this function plays a + * critical role as well. + * + * In a nutshell, this function returns true only if idling is + * beneficial for throughput or, even if detrimental for throughput, + * idling is however necessary to preserve service guarantees (low + * latency, desired throughput distribution, ...). In particular, on + * NCQ-capable devices, this function tries to return false, so as to + * help keep the drives' internal queues full, whenever this helps the + * device boost the throughput without causing any service-guarantee + * issue. + * + * In more detail, the return value of this function is obtained by, + * first, computing a number of boolean variables that take into + * account throughput and service-guarantee issues, and, then, + * combining these variables in a logical expression. Most of the + * issues taken into account are not trivial. We discuss these issues + * while introducing the variables. + */ +static bool bfq_bfqq_may_idle(struct bfq_queue *bfqq) +{ + struct bfq_data *bfqd = bfqq->bfqd; + bool rot_without_queueing = + !blk_queue_nonrot(bfqd->queue) && !bfqd->hw_tag, + bfqq_sequential_and_IO_bound, + idling_boosts_thr, idling_boosts_thr_without_issues, + idling_needed_for_service_guarantees, + asymmetric_scenario; + + if (bfqd->strict_guarantees) + return true; + + /* + * Idling is performed only if slice_idle > 0. In addition, we + * do not idle if + * (a) bfqq is async + * (b) bfqq is in the idle io prio class: in this case we do + * not idle because we want to minimize the bandwidth that + * queues in this class can steal to higher-priority queues + */ + if (bfqd->bfq_slice_idle == 0 || !bfq_bfqq_sync(bfqq) || + bfq_class_idle(bfqq)) + return false; + + bfqq_sequential_and_IO_bound = !BFQQ_SEEKY(bfqq) && + bfq_bfqq_IO_bound(bfqq) && bfq_bfqq_has_short_ttime(bfqq); + /* + * The next variable takes into account the cases where idling + * boosts the throughput. + * + * The value of the variable is computed considering, first, that + * idling is virtually always beneficial for the throughput if: + * (a) the device is not NCQ-capable and rotational, or + * (b) regardless of the presence of NCQ, the device is rotational and + * the request pattern for bfqq is I/O-bound and sequential, or + * (c) regardless of whether it is rotational, the device is + * not NCQ-capable and the request pattern for bfqq is + * I/O-bound and sequential. + * + * Secondly, and in contrast to the above item (b), idling an + * NCQ-capable flash-based device would not boost the + * throughput even with sequential I/O; rather it would lower + * the throughput in proportion to how fast the device + * is. Accordingly, the next variable is true if any of the + * above conditions (a), (b) or (c) is true, and, in + * particular, happens to be false if bfqd is an NCQ-capable + * flash-based device. + */ + idling_boosts_thr = rot_without_queueing || + ((!blk_queue_nonrot(bfqd->queue) || !bfqd->hw_tag) && + bfqq_sequential_and_IO_bound); + + /* + * The value of the next variable, + * idling_boosts_thr_without_issues, is equal to that of + * idling_boosts_thr, unless a special case holds. In this + * special case, described below, idling may cause problems to + * weight-raised queues. + * + * When the request pool is saturated (e.g., in the presence + * of write hogs), if the processes associated with + * non-weight-raised queues ask for requests at a lower rate, + * then processes associated with weight-raised queues have a + * higher probability to get a request from the pool + * immediately (or at least soon) when they need one. Thus + * they have a higher probability to actually get a fraction + * of the device throughput proportional to their high + * weight. This is especially true with NCQ-capable drives, + * which enqueue several requests in advance, and further + * reorder internally-queued requests. + * + * For this reason, we force to false the value of + * idling_boosts_thr_without_issues if there are weight-raised + * busy queues. In this case, and if bfqq is not weight-raised, + * this guarantees that the device is not idled for bfqq (if, + * instead, bfqq is weight-raised, then idling will be + * guaranteed by another variable, see below). Combined with + * the timestamping rules of BFQ (see [1] for details), this + * behavior causes bfqq, and hence any sync non-weight-raised + * queue, to get a lower number of requests served, and thus + * to ask for a lower number of requests from the request + * pool, before the busy weight-raised queues get served + * again. This often mitigates starvation problems in the + * presence of heavy write workloads and NCQ, thereby + * guaranteeing a higher application and system responsiveness + * in these hostile scenarios. + */ + idling_boosts_thr_without_issues = idling_boosts_thr && + bfqd->wr_busy_queues == 0; + + /* + * There is then a case where idling must be performed not + * for throughput concerns, but to preserve service + * guarantees. + * + * To introduce this case, we can note that allowing the drive + * to enqueue more than one request at a time, and hence + * delegating de facto final scheduling decisions to the + * drive's internal scheduler, entails loss of control on the + * actual request service order. In particular, the critical + * situation is when requests from different processes happen + * to be present, at the same time, in the internal queue(s) + * of the drive. In such a situation, the drive, by deciding + * the service order of the internally-queued requests, does + * determine also the actual throughput distribution among + * these processes. But the drive typically has no notion or + * concern about per-process throughput distribution, and + * makes its decisions only on a per-request basis. Therefore, + * the service distribution enforced by the drive's internal + * scheduler is likely to coincide with the desired + * device-throughput distribution only in a completely + * symmetric scenario where: + * (i) each of these processes must get the same throughput as + * the others; + * (ii) all these processes have the same I/O pattern + * (either sequential or random). + * In fact, in such a scenario, the drive will tend to treat + * the requests of each of these processes in about the same + * way as the requests of the others, and thus to provide + * each of these processes with about the same throughput + * (which is exactly the desired throughput distribution). In + * contrast, in any asymmetric scenario, device idling is + * certainly needed to guarantee that bfqq receives its + * assigned fraction of the device throughput (see [1] for + * details). + * + * We address this issue by controlling, actually, only the + * symmetry sub-condition (i), i.e., provided that + * sub-condition (i) holds, idling is not performed, + * regardless of whether sub-condition (ii) holds. In other + * words, only if sub-condition (i) holds, then idling is + * allowed, and the device tends to be prevented from queueing + * many requests, possibly of several processes. The reason + * for not controlling also sub-condition (ii) is that we + * exploit preemption to preserve guarantees in case of + * symmetric scenarios, even if (ii) does not hold, as + * explained in the next two paragraphs. + * + * Even if a queue, say Q, is expired when it remains idle, Q + * can still preempt the new in-service queue if the next + * request of Q arrives soon (see the comments on + * bfq_bfqq_update_budg_for_activation). If all queues and + * groups have the same weight, this form of preemption, + * combined with the hole-recovery heuristic described in the + * comments on function bfq_bfqq_update_budg_for_activation, + * are enough to preserve a correct bandwidth distribution in + * the mid term, even without idling. In fact, even if not + * idling allows the internal queues of the device to contain + * many requests, and thus to reorder requests, we can rather + * safely assume that the internal scheduler still preserves a + * minimum of mid-term fairness. The motivation for using + * preemption instead of idling is that, by not idling, + * service guarantees are preserved without minimally + * sacrificing throughput. In other words, both a high + * throughput and its desired distribution are obtained. + * + * More precisely, this preemption-based, idleless approach + * provides fairness in terms of IOPS, and not sectors per + * second. This can be seen with a simple example. Suppose + * that there are two queues with the same weight, but that + * the first queue receives requests of 8 sectors, while the + * second queue receives requests of 1024 sectors. In + * addition, suppose that each of the two queues contains at + * most one request at a time, which implies that each queue + * always remains idle after it is served. Finally, after + * remaining idle, each queue receives very quickly a new + * request. It follows that the two queues are served + * alternatively, preempting each other if needed. This + * implies that, although both queues have the same weight, + * the queue with large requests receives a service that is + * 1024/8 times as high as the service received by the other + * queue. + * + * On the other hand, device idling is performed, and thus + * pure sector-domain guarantees are provided, for the + * following queues, which are likely to need stronger + * throughput guarantees: weight-raised queues, and queues + * with a higher weight than other queues. When such queues + * are active, sub-condition (i) is false, which triggers + * device idling. + * + * According to the above considerations, the next variable is + * true (only) if sub-condition (i) holds. To compute the + * value of this variable, we not only use the return value of + * the function bfq_symmetric_scenario(), but also check + * whether bfqq is being weight-raised, because + * bfq_symmetric_scenario() does not take into account also + * weight-raised queues (see comments on + * bfq_weights_tree_add()). + * + * As a side note, it is worth considering that the above + * device-idling countermeasures may however fail in the + * following unlucky scenario: if idling is (correctly) + * disabled in a time period during which all symmetry + * sub-conditions hold, and hence the device is allowed to + * enqueue many requests, but at some later point in time some + * sub-condition stops to hold, then it may become impossible + * to let requests be served in the desired order until all + * the requests already queued in the device have been served. + */ + asymmetric_scenario = bfqq->wr_coeff > 1 || + !bfq_symmetric_scenario(bfqd); + + /* + * Finally, there is a case where maximizing throughput is the + * best choice even if it may cause unfairness toward + * bfqq. Such a case is when bfqq became active in a burst of + * queue activations. Queues that became active during a large + * burst benefit only from throughput, as discussed in the + * comments on bfq_handle_burst. Thus, if bfqq became active + * in a burst and not idling the device maximizes throughput, + * then the device must no be idled, because not idling the + * device provides bfqq and all other queues in the burst with + * maximum benefit. Combining this and the above case, we can + * now establish when idling is actually needed to preserve + * service guarantees. + */ + idling_needed_for_service_guarantees = + asymmetric_scenario && !bfq_bfqq_in_large_burst(bfqq); + + /* + * We have now all the components we need to compute the + * return value of the function, which is true only if idling + * either boosts the throughput (without issues), or is + * necessary to preserve service guarantees. + */ + bfq_log_bfqq(bfqd, bfqq, "may_idle: sync %d idling_boosts_thr %d", + bfq_bfqq_sync(bfqq), idling_boosts_thr); + + bfq_log_bfqq(bfqd, bfqq, + "may_idle: wr_busy %d boosts %d IO-bound %d guar %d", + bfqd->wr_busy_queues, + idling_boosts_thr_without_issues, + bfq_bfqq_IO_bound(bfqq), + idling_needed_for_service_guarantees); + + return idling_boosts_thr_without_issues || + idling_needed_for_service_guarantees; +} + +/* + * If the in-service queue is empty but the function bfq_bfqq_may_idle + * returns true, then: + * 1) the queue must remain in service and cannot be expired, and + * 2) the device must be idled to wait for the possible arrival of a new + * request for the queue. + * See the comments on the function bfq_bfqq_may_idle for the reasons + * why performing device idling is the best choice to boost the throughput + * and preserve service guarantees when bfq_bfqq_may_idle itself + * returns true. + */ +static bool bfq_bfqq_must_idle(struct bfq_queue *bfqq) +{ + return RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_bfqq_may_idle(bfqq); +} + +/* + * Select a queue for service. If we have a current queue in service, + * check whether to continue servicing it, or retrieve and set a new one. + */ +static struct bfq_queue *bfq_select_queue(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq; + struct request *next_rq; + enum bfqq_expiration reason = BFQ_BFQQ_BUDGET_TIMEOUT; + + bfqq = bfqd->in_service_queue; + if (!bfqq) + goto new_queue; + + bfq_log_bfqq(bfqd, bfqq, "select_queue: already in-service queue"); + + if (bfq_may_expire_for_budg_timeout(bfqq) && + !bfq_bfqq_wait_request(bfqq) && + !bfq_bfqq_must_idle(bfqq)) + goto expire; + +check_queue: + /* + * This loop is rarely executed more than once. Even when it + * happens, it is much more convenient to re-execute this loop + * than to return NULL and trigger a new dispatch to get a + * request served. + */ + next_rq = bfqq->next_rq; + /* + * If bfqq has requests queued and it has enough budget left to + * serve them, keep the queue, otherwise expire it. + */ + if (next_rq) { + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + + if (bfq_serv_to_charge(next_rq, bfqq) > + bfq_bfqq_budget_left(bfqq)) { + /* + * Expire the queue for budget exhaustion, + * which makes sure that the next budget is + * enough to serve the next request, even if + * it comes from the fifo expired path. + */ + reason = BFQ_BFQQ_BUDGET_EXHAUSTED; + goto expire; + } else { + /* + * The idle timer may be pending because we may + * not disable disk idling even when a new request + * arrives. + */ + if (bfq_bfqq_wait_request(bfqq)) { + /* + * If we get here: 1) at least a new request + * has arrived but we have not disabled the + * timer because the request was too small, + * 2) then the block layer has unplugged + * the device, causing the dispatch to be + * invoked. + * + * Since the device is unplugged, now the + * requests are probably large enough to + * provide a reasonable throughput. + * So we disable idling. + */ + bfq_clear_bfqq_wait_request(bfqq); + hrtimer_try_to_cancel(&bfqd->idle_slice_timer); + bfqg_stats_update_idle_time(bfqq_group(bfqq)); + } + goto keep_queue; + } + } + + /* + * No requests pending. However, if the in-service queue is idling + * for a new request, or has requests waiting for a completion and + * may idle after their completion, then keep it anyway. + */ + if (bfq_bfqq_wait_request(bfqq) || + (bfqq->dispatched != 0 && bfq_bfqq_may_idle(bfqq))) { + bfqq = NULL; + goto keep_queue; + } + + reason = BFQ_BFQQ_NO_MORE_REQUESTS; +expire: + bfq_bfqq_expire(bfqd, bfqq, false, reason); +new_queue: + bfqq = bfq_set_in_service_queue(bfqd); + if (bfqq) { + bfq_log_bfqq(bfqd, bfqq, "select_queue: checking new queue"); + goto check_queue; + } +keep_queue: + if (bfqq) + bfq_log_bfqq(bfqd, bfqq, "select_queue: returned this queue"); + else + bfq_log(bfqd, "select_queue: no queue returned"); + + return bfqq; +} + +static void bfq_update_wr_data(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + + if (bfqq->wr_coeff > 1) { /* queue is being weight-raised */ + BUG_ON(bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time && + time_is_after_jiffies(bfqq->last_wr_start_finish)); + + bfq_log_bfqq(bfqd, bfqq, + "raising period dur %u/%u msec, old coeff %u, w %d(%d)", + jiffies_to_msecs(jiffies - bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time), + bfqq->wr_coeff, + bfqq->entity.weight, bfqq->entity.orig_weight); + + BUG_ON(bfqq != bfqd->in_service_queue && entity->weight != + entity->orig_weight * bfqq->wr_coeff); + if (entity->prio_changed) + bfq_log_bfqq(bfqd, bfqq, "WARN: pending prio change"); + + /* + * If the queue was activated in a burst, or too much + * time has elapsed from the beginning of this + * weight-raising period, then end weight raising. + */ + if (bfq_bfqq_in_large_burst(bfqq)) + bfq_bfqq_end_wr(bfqq); + else if (time_is_before_jiffies(bfqq->last_wr_start_finish + + bfqq->wr_cur_max_time)) { + if (bfqq->wr_cur_max_time != bfqd->bfq_wr_rt_max_time || + time_is_before_jiffies(bfqq->wr_start_at_switch_to_srt + + bfq_wr_duration(bfqd))) + bfq_bfqq_end_wr(bfqq); + else { + /* switch back to interactive wr */ + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + bfqq->last_wr_start_finish = + bfqq->wr_start_at_switch_to_srt; + BUG_ON(time_is_after_jiffies( + bfqq->last_wr_start_finish)); + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqd, bfqq, + "back to interactive wr"); + } + } + } + /* + * To improve latency (for this or other queues), immediately + * update weight both if it must be raised and if it must be + * lowered. Since, entity may be on some active tree here, and + * might have a pending change of its ioprio class, invoke + * next function with the last parameter unset (see the + * comments on the function). + */ + if ((entity->weight > entity->orig_weight) != (bfqq->wr_coeff > 1)) + __bfq_entity_update_weight_prio(bfq_entity_service_tree(entity), + entity, false); +} + +/* + * Dispatch next request from bfqq. + */ +static struct request *bfq_dispatch_rq_from_bfqq(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + struct request *rq = bfqq->next_rq; + unsigned long service_to_charge; + + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + BUG_ON(!rq); + service_to_charge = bfq_serv_to_charge(rq, bfqq); + + BUG_ON(service_to_charge > bfq_bfqq_budget_left(bfqq)); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + bfq_bfqq_served(bfqq, service_to_charge); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + bfq_dispatch_remove(bfqd->queue, rq); + + /* + * If weight raising has to terminate for bfqq, then next + * function causes an immediate update of bfqq's weight, + * without waiting for next activation. As a consequence, on + * expiration, bfqq will be timestamped as if has never been + * weight-raised during this service slot, even if it has + * received part or even most of the service as a + * weight-raised queue. This inflates bfqq's timestamps, which + * is beneficial, as bfqq is then more willing to leave the + * device immediately to possible other weight-raised queues. + */ + bfq_update_wr_data(bfqd, bfqq); + + bfq_log_bfqq(bfqd, bfqq, + "dispatched %u sec req (%llu), budg left %d, new disp_nr %d", + blk_rq_sectors(rq), + (unsigned long long) blk_rq_pos(rq), + bfq_bfqq_budget_left(bfqq), + bfqq->dispatched); + + /* + * Expire bfqq, pretending that its budget expired, if bfqq + * belongs to CLASS_IDLE and other queues are waiting for + * service. + */ + if (bfqd->busy_queues > 1 && bfq_class_idle(bfqq)) + goto expire; + + return rq; + +expire: + bfq_bfqq_expire(bfqd, bfqq, false, BFQ_BFQQ_BUDGET_EXHAUSTED); + return rq; +} + +static bool bfq_has_work(struct blk_mq_hw_ctx *hctx) +{ + struct bfq_data *bfqd = hctx->queue->elevator->elevator_data; + + bfq_log(bfqd, "has_work, dispatch_non_empty %d busy_queues %d", + !list_empty_careful(&bfqd->dispatch), bfqd->busy_queues > 0); + + /* + * Avoiding lock: a race on bfqd->busy_queues should cause at + * most a call to dispatch for nothing + */ + return !list_empty_careful(&bfqd->dispatch) || + bfqd->busy_queues > 0; +} + +static struct request *__bfq_dispatch_request(struct blk_mq_hw_ctx *hctx) +{ + struct bfq_data *bfqd = hctx->queue->elevator->elevator_data; + struct request *rq = NULL; + struct bfq_queue *bfqq = NULL; + + if (!list_empty(&bfqd->dispatch)) { + rq = list_first_entry(&bfqd->dispatch, struct request, + queuelist); + list_del_init(&rq->queuelist); + rq->rq_flags &= ~RQF_DISP_LIST; + + bfq_log(bfqd, + "dispatch requests: picked %p from dispatch list", rq); + bfqq = RQ_BFQQ(rq); + + if (bfqq) { + /* + * Increment counters here, because this + * dispatch does not follow the standard + * dispatch flow (where counters are + * incremented) + */ + bfqq->dispatched++; + + /* + * TESTING: reset DISP_LIST flag, because: 1) + * this rq this request has passed through + * get_rq_private, 2) then it will have + * put_rq_private invoked on it, and 3) in + * put_rq_private we use this flag to check + * that put_rq_private is not invoked on + * requests for which get_rq_private has been + * invoked. + */ + rq->rq_flags &= ~RQF_DISP_LIST; + goto inc_in_driver_start_rq; + } + + /* + * We exploit the put_rq_private hook to decrement + * rq_in_driver, but put_rq_private will not be + * invoked on this request. So, to avoid unbalance, + * just start this request, without incrementing + * rq_in_driver. As a negative consequence, + * rq_in_driver is deceptively lower than it should be + * while this request is in service. This may cause + * bfq_schedule_dispatch to be invoked uselessly. + * + * As for implementing an exact solution, the + * put_request hook, if defined, is probably invoked + * also on this request. So, by exploiting this hook, + * we could 1) increment rq_in_driver here, and 2) + * decrement it in put_request. Such a solution would + * let the value of the counter be always accurate, + * but it would entail using an extra interface + * function. This cost seems higher than the benefit, + * being the frequency of non-elevator-private + * requests very low. + */ + goto start_rq; + } + + bfq_log(bfqd, "dispatch requests: %d busy queues", bfqd->busy_queues); + + if (bfqd->busy_queues == 0) + goto exit; + + /* + * Force device to serve one request at a time if + * strict_guarantees is true. Forcing this service scheme is + * currently the ONLY way to guarantee that the request + * service order enforced by the scheduler is respected by a + * queueing device. Otherwise the device is free even to make + * some unlucky request wait for as long as the device + * wishes. + * + * Of course, serving one request at at time may cause loss of + * throughput. + */ + if (bfqd->strict_guarantees && bfqd->rq_in_driver > 0) + goto exit; + + bfqq = bfq_select_queue(bfqd); + if (!bfqq) + goto exit; + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + BUG_ON(bfq_bfqq_wait_request(bfqq)); + + rq = bfq_dispatch_rq_from_bfqq(bfqd, bfqq); + + BUG_ON(bfqq->next_rq == NULL && + bfqq->entity.budget < bfqq->entity.service); + + if (rq) { + inc_in_driver_start_rq: + bfqd->rq_in_driver++; + start_rq: + rq->rq_flags |= RQF_STARTED; + if (bfqq) + bfq_log_bfqq(bfqd, bfqq, + "dispatched %s request %p, rq_in_driver %d", + bfq_bfqq_sync(bfqq) ? "sync" : "async", + rq, + bfqd->rq_in_driver); + else + bfq_log(bfqd, + "dispatched request %p from dispatch list, rq_in_driver %d", + rq, bfqd->rq_in_driver); + } else + bfq_log(bfqd, + "returned NULL request, rq_in_driver %d", + bfqd->rq_in_driver); + +exit: + return rq; +} + +static struct request *bfq_dispatch_request(struct blk_mq_hw_ctx *hctx) +{ + struct bfq_data *bfqd = hctx->queue->elevator->elevator_data; + struct request *rq; + + spin_lock_irq(&bfqd->lock); + + rq = __bfq_dispatch_request(hctx); + spin_unlock_irq(&bfqd->lock); + + return rq; +} + +/* + * Task holds one reference to the queue, dropped when task exits. Each rq + * in-flight on this queue also holds a reference, dropped when rq is freed. + * + * Scheduler lock must be held here. Recall not to use bfqq after calling + * this function on it. + */ +static void bfq_put_queue(struct bfq_queue *bfqq) +{ +#ifdef BFQ_GROUP_IOSCHED_ENABLED + struct bfq_group *bfqg = bfqq_group(bfqq); +#endif + + assert_spin_locked(&bfqq->bfqd->lock); + + BUG_ON(bfqq->ref <= 0); + + if (bfqq->bfqd) + bfq_log_bfqq(bfqq->bfqd, bfqq, "put_queue: %p %d", bfqq, bfqq->ref); + + bfqq->ref--; + if (bfqq->ref) + return; + + BUG_ON(rb_first(&bfqq->sort_list)); + BUG_ON(bfqq->allocated != 0); + BUG_ON(bfqq->entity.tree); + BUG_ON(bfq_bfqq_busy(bfqq)); + + if (bfq_bfqq_sync(bfqq)) + /* + * The fact that this queue is being destroyed does not + * invalidate the fact that this queue may have been + * activated during the current burst. As a consequence, + * although the queue does not exist anymore, and hence + * needs to be removed from the burst list if there, + * the burst size has not to be decremented. + */ + hlist_del_init(&bfqq->burst_list_node); + + if (bfqq->bfqd) + bfq_log_bfqq(bfqq->bfqd, bfqq, "put_queue: %p freed", bfqq); + + kmem_cache_free(bfq_pool, bfqq); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + bfqg_and_blkg_put(bfqg); +#endif +} + +static void bfq_put_cooperator(struct bfq_queue *bfqq) +{ + struct bfq_queue *__bfqq, *next; + + /* + * If this queue was scheduled to merge with another queue, be + * sure to drop the reference taken on that queue (and others in + * the merge chain). See bfq_setup_merge and bfq_merge_bfqqs. + */ + __bfqq = bfqq->new_bfqq; + while (__bfqq) { + if (__bfqq == bfqq) + break; + next = __bfqq->new_bfqq; + bfq_put_queue(__bfqq); + __bfqq = next; + } +} + +static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + if (bfqq == bfqd->in_service_queue) { + __bfq_bfqq_expire(bfqd, bfqq); + bfq_schedule_dispatch(bfqd); + } + + bfq_log_bfqq(bfqd, bfqq, "exit_bfqq: %p, %d", bfqq, bfqq->ref); + + bfq_put_cooperator(bfqq); + + bfq_put_queue(bfqq); /* release process reference */ +} + +static void bfq_exit_icq_bfqq(struct bfq_io_cq *bic, bool is_sync) +{ + struct bfq_queue *bfqq = bic_to_bfqq(bic, is_sync); + struct bfq_data *bfqd; + + if (bfqq) + bfqd = bfqq->bfqd; /* NULL if scheduler already exited */ + + if (bfqq && bfqd) { + unsigned long flags; + + spin_lock_irqsave(&bfqd->lock, flags); + + bfq_exit_bfqq(bfqd, bfqq); + bic_set_bfqq(bic, NULL, is_sync); + spin_unlock_irqrestore(&bfqd->lock, flags); + } +} + +static void bfq_exit_icq(struct io_cq *icq) +{ + struct bfq_io_cq *bic = icq_to_bic(icq); + + BUG_ON(!bic); + bfq_exit_icq_bfqq(bic, true); + bfq_exit_icq_bfqq(bic, false); +} + +/* + * Update the entity prio values; note that the new values will not + * be used until the next (re)activation. + */ +static void bfq_set_next_ioprio_data(struct bfq_queue *bfqq, + struct bfq_io_cq *bic) +{ + struct task_struct *tsk = current; + int ioprio_class; + struct bfq_data *bfqd = bfqq->bfqd; + + WARN_ON(!bfqd); + if (!bfqd) + return; + + ioprio_class = IOPRIO_PRIO_CLASS(bic->ioprio); + switch (ioprio_class) { + default: + dev_err(bfqq->bfqd->queue->backing_dev_info->dev, + "bfq: bad prio class %d\n", ioprio_class); + case IOPRIO_CLASS_NONE: + /* + * No prio set, inherit CPU scheduling settings. + */ + bfqq->new_ioprio = task_nice_ioprio(tsk); + bfqq->new_ioprio_class = task_nice_ioclass(tsk); + break; + case IOPRIO_CLASS_RT: + bfqq->new_ioprio = IOPRIO_PRIO_DATA(bic->ioprio); + bfqq->new_ioprio_class = IOPRIO_CLASS_RT; + break; + case IOPRIO_CLASS_BE: + bfqq->new_ioprio = IOPRIO_PRIO_DATA(bic->ioprio); + bfqq->new_ioprio_class = IOPRIO_CLASS_BE; + break; + case IOPRIO_CLASS_IDLE: + bfqq->new_ioprio_class = IOPRIO_CLASS_IDLE; + bfqq->new_ioprio = 7; + break; + } + + if (bfqq->new_ioprio >= IOPRIO_BE_NR) { + pr_crit("bfq_set_next_ioprio_data: new_ioprio %d\n", + bfqq->new_ioprio); + BUG(); + } + + bfqq->entity.new_weight = bfq_ioprio_to_weight(bfqq->new_ioprio); + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqq->bfqd, bfqq, + "set_next_ioprio_data: bic_class %d prio %d class %d", + ioprio_class, bfqq->new_ioprio, bfqq->new_ioprio_class); +} + +static void bfq_check_ioprio_change(struct bfq_io_cq *bic, struct bio *bio) +{ + struct bfq_data *bfqd = bic_to_bfqd(bic); + struct bfq_queue *bfqq; + unsigned long uninitialized_var(flags); + int ioprio = bic->icq.ioc->ioprio; + + /* + * This condition may trigger on a newly created bic, be sure to + * drop the lock before returning. + */ + if (unlikely(!bfqd) || likely(bic->ioprio == ioprio)) + return; + + bic->ioprio = ioprio; + + bfqq = bic_to_bfqq(bic, false); + if (bfqq) { + /* release process reference on this queue */ + bfq_put_queue(bfqq); + bfqq = bfq_get_queue(bfqd, bio, BLK_RW_ASYNC, bic); + bic_set_bfqq(bic, bfqq, false); + bfq_log_bfqq(bfqd, bfqq, + "check_ioprio_change: bfqq %p %d", + bfqq, bfqq->ref); + } + + bfqq = bic_to_bfqq(bic, true); + if (bfqq) + bfq_set_next_ioprio_data(bfqq, bic); +} + +static void bfq_init_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct bfq_io_cq *bic, pid_t pid, int is_sync) +{ + RB_CLEAR_NODE(&bfqq->entity.rb_node); + INIT_LIST_HEAD(&bfqq->fifo); + INIT_HLIST_NODE(&bfqq->burst_list_node); + BUG_ON(!hlist_unhashed(&bfqq->burst_list_node)); + + bfqq->ref = 0; + bfqq->bfqd = bfqd; + + if (bic) + bfq_set_next_ioprio_data(bfqq, bic); + + if (is_sync) { + /* + * No need to mark as has_short_ttime if in + * idle_class, because no device idling is performed + * for queues in idle class + */ + if (!bfq_class_idle(bfqq)) + /* tentatively mark as has_short_ttime */ + bfq_mark_bfqq_has_short_ttime(bfqq); + bfq_mark_bfqq_sync(bfqq); + bfq_mark_bfqq_just_created(bfqq); + } else + bfq_clear_bfqq_sync(bfqq); + + bfqq->ttime.last_end_request = ktime_get_ns() - (1ULL<<32); + + bfq_mark_bfqq_IO_bound(bfqq); + + /* Tentative initial value to trade off between thr and lat */ + bfqq->max_budget = (2 * bfq_max_budget(bfqd)) / 3; + bfqq->pid = pid; + + bfqq->wr_coeff = 1; + bfqq->last_wr_start_finish = jiffies; + bfqq->wr_start_at_switch_to_srt = bfq_smallest_from_now(); + bfqq->budget_timeout = bfq_smallest_from_now(); + bfqq->split_time = bfq_smallest_from_now(); + + /* + * Set to the value for which bfqq will not be deemed as + * soft rt when it becomes backlogged. + */ + bfqq->soft_rt_next_start = bfq_greatest_from_now(); + + /* first request is almost certainly seeky */ + bfqq->seek_history = 1; +} + +static struct bfq_queue **bfq_async_queue_prio(struct bfq_data *bfqd, + struct bfq_group *bfqg, + int ioprio_class, int ioprio) +{ + switch (ioprio_class) { + case IOPRIO_CLASS_RT: + return &bfqg->async_bfqq[0][ioprio]; + case IOPRIO_CLASS_NONE: + ioprio = IOPRIO_NORM; + /* fall through */ + case IOPRIO_CLASS_BE: + return &bfqg->async_bfqq[1][ioprio]; + case IOPRIO_CLASS_IDLE: + return &bfqg->async_idle_bfqq; + default: + BUG(); + } +} + +static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, + struct bio *bio, bool is_sync, + struct bfq_io_cq *bic) +{ + const int ioprio = IOPRIO_PRIO_DATA(bic->ioprio); + const int ioprio_class = IOPRIO_PRIO_CLASS(bic->ioprio); + struct bfq_queue **async_bfqq = NULL; + struct bfq_queue *bfqq; + struct bfq_group *bfqg; + + rcu_read_lock(); + + bfqg = bfq_find_set_group(bfqd, bio_blkcg(bio)); + if (!bfqg) { + bfqq = &bfqd->oom_bfqq; + goto out; + } + + if (!is_sync) { + async_bfqq = bfq_async_queue_prio(bfqd, bfqg, ioprio_class, + ioprio); + bfqq = *async_bfqq; + if (bfqq) + goto out; + } + + bfqq = kmem_cache_alloc_node(bfq_pool, + GFP_NOWAIT | __GFP_ZERO | __GFP_NOWARN, + bfqd->queue->node); + + if (bfqq) { + bfq_init_bfqq(bfqd, bfqq, bic, current->pid, + is_sync); + bfq_init_entity(&bfqq->entity, bfqg); + bfq_log_bfqq(bfqd, bfqq, "allocated"); + } else { + bfqq = &bfqd->oom_bfqq; + bfq_log_bfqq(bfqd, bfqq, "using oom bfqq"); + goto out; + } + + /* + * Pin the queue now that it's allocated, scheduler exit will + * prune it. + */ + if (async_bfqq) { + bfqq->ref++; /* + * Extra group reference, w.r.t. sync + * queue. This extra reference is removed + * only if bfqq->bfqg disappears, to + * guarantee that this queue is not freed + * until its group goes away. + */ + bfq_log_bfqq(bfqd, bfqq, "get_queue, bfqq not in async: %p, %d", + bfqq, bfqq->ref); + *async_bfqq = bfqq; + } + +out: + bfqq->ref++; /* get a process reference to this queue */ + bfq_log_bfqq(bfqd, bfqq, "get_queue, at end: %p, %d", bfqq, bfqq->ref); + rcu_read_unlock(); + return bfqq; +} + +static void bfq_update_io_thinktime(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + struct bfq_ttime *ttime = &bfqq->ttime; + u64 elapsed = ktime_get_ns() - bfqq->ttime.last_end_request; + + elapsed = min_t(u64, elapsed, 2 * bfqd->bfq_slice_idle); + + ttime->ttime_samples = (7*bfqq->ttime.ttime_samples + 256) / 8; + ttime->ttime_total = div_u64(7*ttime->ttime_total + 256*elapsed, 8); + ttime->ttime_mean = div64_ul(ttime->ttime_total + 128, + ttime->ttime_samples); +} + +static void +bfq_update_io_seektime(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct request *rq) +{ + bfqq->seek_history <<= 1; + bfqq->seek_history |= + get_sdist(bfqq->last_request_pos, rq) > BFQQ_SEEK_THR && + (!blk_queue_nonrot(bfqd->queue) || + blk_rq_sectors(rq) < BFQQ_SECT_THR_NONROT); +} + +static void bfq_update_has_short_ttime(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + struct bfq_io_cq *bic) +{ + bool has_short_ttime = true; + + /* + * No need to update has_short_ttime if bfqq is async or in + * idle io prio class, or if bfq_slice_idle is zero, because + * no device idling is performed for bfqq in this case. + */ + if (!bfq_bfqq_sync(bfqq) || bfq_class_idle(bfqq) || + bfqd->bfq_slice_idle == 0) + return; + + /* Idle window just restored, statistics are meaningless. */ + if (time_is_after_eq_jiffies(bfqq->split_time + + bfqd->bfq_wr_min_idle_time)) + return; + + /* Think time is infinite if no process is linked to + * bfqq. Otherwise check average think time to + * decide whether to mark as has_short_ttime + */ + if (atomic_read(&bic->icq.ioc->active_ref) == 0 || + (bfq_sample_valid(bfqq->ttime.ttime_samples) && + bfqq->ttime.ttime_mean > bfqd->bfq_slice_idle)) + has_short_ttime = false; + + bfq_log_bfqq(bfqd, bfqq, "update_has_short_ttime: has_short_ttime %d", + has_short_ttime); + + if (has_short_ttime) + bfq_mark_bfqq_has_short_ttime(bfqq); + else + bfq_clear_bfqq_has_short_ttime(bfqq); +} + +/* + * Called when a new fs request (rq) is added to bfqq. Check if there's + * something we should do about it. + */ +static void bfq_rq_enqueued(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct request *rq) +{ + struct bfq_io_cq *bic = RQ_BIC(rq); + + if (rq->cmd_flags & REQ_META) + bfqq->meta_pending++; + + bfq_update_io_thinktime(bfqd, bfqq); + bfq_update_has_short_ttime(bfqd, bfqq, bic); + bfq_update_io_seektime(bfqd, bfqq, rq); + + bfq_log_bfqq(bfqd, bfqq, + "rq_enqueued: has_short_ttime=%d (seeky %d)", + bfq_bfqq_has_short_ttime(bfqq), BFQQ_SEEKY(bfqq)); + + bfqq->last_request_pos = blk_rq_pos(rq) + blk_rq_sectors(rq); + + if (bfqq == bfqd->in_service_queue && bfq_bfqq_wait_request(bfqq)) { + bool small_req = bfqq->queued[rq_is_sync(rq)] == 1 && + blk_rq_sectors(rq) < 32; + bool budget_timeout = bfq_bfqq_budget_timeout(bfqq); + + /* + * There is just this request queued: if the request + * is small and the queue is not to be expired, then + * just exit. + * + * In this way, if the device is being idled to wait + * for a new request from the in-service queue, we + * avoid unplugging the device and committing the + * device to serve just a small request. On the + * contrary, we wait for the block layer to decide + * when to unplug the device: hopefully, new requests + * will be merged to this one quickly, then the device + * will be unplugged and larger requests will be + * dispatched. + */ + if (small_req && !budget_timeout) + return; + + /* + * A large enough request arrived, or the queue is to + * be expired: in both cases disk idling is to be + * stopped, so clear wait_request flag and reset + * timer. + */ + bfq_clear_bfqq_wait_request(bfqq); + hrtimer_try_to_cancel(&bfqd->idle_slice_timer); + bfqg_stats_update_idle_time(bfqq_group(bfqq)); + + /* + * The queue is not empty, because a new request just + * arrived. Hence we can safely expire the queue, in + * case of budget timeout, without risking that the + * timestamps of the queue are not updated correctly. + * See [1] for more details. + */ + if (budget_timeout) + bfq_bfqq_expire(bfqd, bfqq, false, + BFQ_BFQQ_BUDGET_TIMEOUT); + } +} + +static void __bfq_insert_request(struct bfq_data *bfqd, struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq), *new_bfqq; + BUG_ON(!bfqq); + + assert_spin_locked(&bfqd->lock); + + bfq_log_bfqq(bfqd, bfqq, "__insert_req: rq %p bfqq %p", rq, bfqq); + + /* + * An unplug may trigger a requeue of a request from the device + * driver: make sure we are in process context while trying to + * merge two bfq_queues. + */ + if (!in_interrupt()) { + new_bfqq = bfq_setup_cooperator(bfqd, bfqq, rq, true); + if (new_bfqq) { + if (bic_to_bfqq(RQ_BIC(rq), 1) != bfqq) + new_bfqq = bic_to_bfqq(RQ_BIC(rq), 1); + /* + * Release the request's reference to the old bfqq + * and make sure one is taken to the shared queue. + */ + new_bfqq->allocated++; + bfqq->allocated--; + bfq_log_bfqq(bfqd, bfqq, + "insert_request: new allocated %d", bfqq->allocated); + bfq_log_bfqq(bfqd, new_bfqq, + "insert_request: new_bfqq new allocated %d", + bfqq->allocated); + + new_bfqq->ref++; + bfq_clear_bfqq_just_created(bfqq); + /* + * If the bic associated with the process + * issuing this request still points to bfqq + * (and thus has not been already redirected + * to new_bfqq or even some other bfq_queue), + * then complete the merge and redirect it to + * new_bfqq. + */ + if (bic_to_bfqq(RQ_BIC(rq), 1) == bfqq) + bfq_merge_bfqqs(bfqd, RQ_BIC(rq), + bfqq, new_bfqq); + /* + * rq is about to be enqueued into new_bfqq, + * release rq reference on bfqq + */ + bfq_put_queue(bfqq); + rq->elv.priv[1] = new_bfqq; + bfqq = new_bfqq; + } + } + + bfq_add_request(rq); + + rq->fifo_time = ktime_get_ns() + bfqd->bfq_fifo_expire[rq_is_sync(rq)]; + list_add_tail(&rq->queuelist, &bfqq->fifo); + + bfq_rq_enqueued(bfqd, bfqq, rq); +} + +static void bfq_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, + bool at_head) +{ + struct request_queue *q = hctx->queue; + struct bfq_data *bfqd = q->elevator->elevator_data; + + spin_lock_irq(&bfqd->lock); + if (blk_mq_sched_try_insert_merge(q, rq)) { + spin_unlock_irq(&bfqd->lock); + return; + } + + spin_unlock_irq(&bfqd->lock); + + blk_mq_sched_request_inserted(rq); + + spin_lock_irq(&bfqd->lock); + if (at_head || blk_rq_is_passthrough(rq)) { + struct bfq_queue *bfqq = RQ_BFQQ(rq); + + if (at_head) + list_add(&rq->queuelist, &bfqd->dispatch); + else + list_add_tail(&rq->queuelist, &bfqd->dispatch); + + rq->rq_flags |= RQF_DISP_LIST; + if (bfqq) + bfq_log_bfqq(bfqd, bfqq, + "insert_request %p in disp: at_head %d", + rq, at_head); + else + bfq_log(bfqd, + "insert_request %p in disp: at_head %d", + rq, at_head); + } else { + BUG_ON(!(rq->rq_flags & RQF_GOT)); + rq->rq_flags &= ~RQF_GOT; + + __bfq_insert_request(bfqd, rq); + + if (rq_mergeable(rq)) { + elv_rqhash_add(q, rq); + if (!q->last_merge) + q->last_merge = rq; + } + } + + spin_unlock_irq(&bfqd->lock); +} + +static void bfq_insert_requests(struct blk_mq_hw_ctx *hctx, + struct list_head *list, bool at_head) +{ + while (!list_empty(list)) { + struct request *rq; + + rq = list_first_entry(list, struct request, queuelist); + list_del_init(&rq->queuelist); + bfq_insert_request(hctx, rq, at_head); + } +} + +static void bfq_update_hw_tag(struct bfq_data *bfqd) +{ + bfqd->max_rq_in_driver = max_t(int, bfqd->max_rq_in_driver, + bfqd->rq_in_driver); + + if (bfqd->hw_tag == 1) + return; + + /* + * This sample is valid if the number of outstanding requests + * is large enough to allow a queueing behavior. Note that the + * sum is not exact, as it's not taking into account deactivated + * requests. + */ + if (bfqd->rq_in_driver + bfqd->queued < BFQ_HW_QUEUE_THRESHOLD) + return; + + if (bfqd->hw_tag_samples++ < BFQ_HW_QUEUE_SAMPLES) + return; + + bfqd->hw_tag = bfqd->max_rq_in_driver > BFQ_HW_QUEUE_THRESHOLD; + bfqd->max_rq_in_driver = 0; + bfqd->hw_tag_samples = 0; +} + +static void bfq_completed_request(struct bfq_queue *bfqq, struct bfq_data *bfqd) +{ + u64 now_ns; + u32 delta_us; + + bfq_update_hw_tag(bfqd); + + BUG_ON(!bfqd->rq_in_driver); + BUG_ON(!bfqq->dispatched); + bfqd->rq_in_driver--; + bfqq->dispatched--; + + bfq_log_bfqq(bfqd, bfqq, + "completed_requests: new disp %d, new rq_in_driver %d", + bfqq->dispatched, bfqd->rq_in_driver); + + if (!bfqq->dispatched && !bfq_bfqq_busy(bfqq)) { + BUG_ON(!RB_EMPTY_ROOT(&bfqq->sort_list)); + /* + * Set budget_timeout (which we overload to store the + * time at which the queue remains with no backlog and + * no outstanding request; used by the weight-raising + * mechanism). + */ + bfqq->budget_timeout = jiffies; + + bfq_weights_tree_remove(bfqd, &bfqq->entity, + &bfqd->queue_weights_tree); + } + + now_ns = ktime_get_ns(); + + bfqq->ttime.last_end_request = now_ns; + + /* + * Using us instead of ns, to get a reasonable precision in + * computing rate in next check. + */ + delta_us = div_u64(now_ns - bfqd->last_completion, NSEC_PER_USEC); + + bfq_log_bfqq(bfqd, bfqq, + "rq_completed: delta %uus/%luus max_size %u rate %llu/%llu", + delta_us, BFQ_MIN_TT/NSEC_PER_USEC, bfqd->last_rq_max_size, + (USEC_PER_SEC* + (u64)((bfqd->last_rq_max_size<>BFQ_RATE_SHIFT, + (USEC_PER_SEC*(u64)(1UL<<(BFQ_RATE_SHIFT-10)))>>BFQ_RATE_SHIFT); + + /* + * If the request took rather long to complete, and, according + * to the maximum request size recorded, this completion latency + * implies that the request was certainly served at a very low + * rate (less than 1M sectors/sec), then the whole observation + * interval that lasts up to this time instant cannot be a + * valid time interval for computing a new peak rate. Invoke + * bfq_update_rate_reset to have the following three steps + * taken: + * - close the observation interval at the last (previous) + * request dispatch or completion + * - compute rate, if possible, for that observation interval + * - reset to zero samples, which will trigger a proper + * re-initialization of the observation interval on next + * dispatch + */ + if (delta_us > BFQ_MIN_TT/NSEC_PER_USEC && + (bfqd->last_rq_max_size<last_completion = now_ns; + + /* + * If we are waiting to discover whether the request pattern + * of the task associated with the queue is actually + * isochronous, and both requisites for this condition to hold + * are now satisfied, then compute soft_rt_next_start (see the + * comments on the function bfq_bfqq_softrt_next_start()). We + * schedule this delayed check when bfqq expires, if it still + * has in-flight requests. + */ + if (bfq_bfqq_softrt_update(bfqq) && bfqq->dispatched == 0 && + RB_EMPTY_ROOT(&bfqq->sort_list)) + bfqq->soft_rt_next_start = + bfq_bfqq_softrt_next_start(bfqd, bfqq); + + /* + * If this is the in-service queue, check if it needs to be expired, + * or if we want to idle in case it has no pending requests. + */ + if (bfqd->in_service_queue == bfqq) { + if (bfqq->dispatched == 0 && bfq_bfqq_must_idle(bfqq)) { + bfq_arm_slice_timer(bfqd); + return; + } else if (bfq_may_expire_for_budg_timeout(bfqq)) + bfq_bfqq_expire(bfqd, bfqq, false, + BFQ_BFQQ_BUDGET_TIMEOUT); + else if (RB_EMPTY_ROOT(&bfqq->sort_list) && + (bfqq->dispatched == 0 || + !bfq_bfqq_may_idle(bfqq))) + bfq_bfqq_expire(bfqd, bfqq, false, + BFQ_BFQQ_NO_MORE_REQUESTS); + } +} + +static void bfq_put_rq_priv_body(struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqq->bfqd, bfqq, + "put_request_body: allocated %d", bfqq->allocated); + BUG_ON(!bfqq->allocated); + bfqq->allocated--; + + bfq_put_queue(bfqq); +} + +static void bfq_finish_request(struct request *rq) +{ + struct bfq_queue *bfqq; + struct bfq_data *bfqd; + struct bfq_io_cq *bic; + + BUG_ON(!rq); + + if (!rq->elv.icq) + return; + + bfqq = RQ_BFQQ(rq); + BUG_ON(!bfqq); + + bic = RQ_BIC(rq); + BUG_ON(!bic); + + bfqd = bfqq->bfqd; + BUG_ON(!bfqd); + + if (rq->rq_flags & RQF_DISP_LIST) { + pr_crit("putting disp rq %p for %d", rq, bfqq->pid); + BUG(); + } + BUG_ON(rq->rq_flags & RQF_QUEUED); + BUG_ON(!(rq->rq_flags & RQF_ELVPRIV)); + + bfq_log_bfqq(bfqd, bfqq, + "putting rq %p with %u sects left, STARTED %d", + rq, blk_rq_sectors(rq), + rq->rq_flags & RQF_STARTED); + + if (rq->rq_flags & RQF_STARTED) + bfqg_stats_update_completion(bfqq_group(bfqq), + rq_start_time_ns(rq), + rq_io_start_time_ns(rq), + rq->cmd_flags); + + BUG_ON(blk_rq_sectors(rq) == 0 && !(rq->rq_flags & RQF_STARTED)); + + if (likely(rq->rq_flags & RQF_STARTED)) { + unsigned long flags; + + spin_lock_irqsave(&bfqd->lock, flags); + + bfq_completed_request(bfqq, bfqd); + bfq_put_rq_priv_body(bfqq); + + spin_unlock_irqrestore(&bfqd->lock, flags); + } else { + /* + * Request rq may be still/already in the scheduler, + * in which case we need to remove it. And we cannot + * defer such a check and removal, to avoid + * inconsistencies in the time interval from the end + * of this function to the start of the deferred work. + * This situation seems to occur only in process + * context, as a consequence of a merge. In the + * current version of the code, this implies that the + * lock is held. + */ + BUG_ON(in_interrupt()); + + assert_spin_locked(&bfqd->lock); + if (!RB_EMPTY_NODE(&rq->rb_node)) + bfq_remove_request(rq->q, rq); + bfq_put_rq_priv_body(bfqq); + } + + rq->elv.priv[0] = NULL; + rq->elv.priv[1] = NULL; +} + +/* + * Returns NULL if a new bfqq should be allocated, or the old bfqq if this + * was the last process referring to that bfqq. + */ +static struct bfq_queue * +bfq_split_bfqq(struct bfq_io_cq *bic, struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqq->bfqd, bfqq, "splitting queue"); + + if (bfqq_process_refs(bfqq) == 1) { + bfqq->pid = current->pid; + bfq_clear_bfqq_coop(bfqq); + bfq_clear_bfqq_split_coop(bfqq); + return bfqq; + } + + bic_set_bfqq(bic, NULL, 1); + + bfq_put_cooperator(bfqq); + + bfq_put_queue(bfqq); + return NULL; +} + +static struct bfq_queue *bfq_get_bfqq_handle_split(struct bfq_data *bfqd, + struct bfq_io_cq *bic, + struct bio *bio, + bool split, bool is_sync, + bool *new_queue) +{ + struct bfq_queue *bfqq = bic_to_bfqq(bic, is_sync); + + if (likely(bfqq && bfqq != &bfqd->oom_bfqq)) + return bfqq; + + if (new_queue) + *new_queue = true; + + if (bfqq) + bfq_put_queue(bfqq); + bfqq = bfq_get_queue(bfqd, bio, is_sync, bic); + BUG_ON(!hlist_unhashed(&bfqq->burst_list_node)); + + bic_set_bfqq(bic, bfqq, is_sync); + if (split && is_sync) { + bfq_log_bfqq(bfqd, bfqq, + "get_request: was_in_list %d " + "was_in_large_burst %d " + "large burst in progress %d", + bic->was_in_burst_list, + bic->saved_in_large_burst, + bfqd->large_burst); + + if ((bic->was_in_burst_list && bfqd->large_burst) || + bic->saved_in_large_burst) { + bfq_log_bfqq(bfqd, bfqq, + "get_request: marking in " + "large burst"); + bfq_mark_bfqq_in_large_burst(bfqq); + } else { + bfq_log_bfqq(bfqd, bfqq, + "get_request: clearing in " + "large burst"); + bfq_clear_bfqq_in_large_burst(bfqq); + if (bic->was_in_burst_list) + hlist_add_head(&bfqq->burst_list_node, + &bfqd->burst_list); + } + bfqq->split_time = jiffies; + } + + return bfqq; +} + +/* + * Allocate bfq data structures associated with this request. + */ +static void bfq_prepare_request(struct request *rq, struct bio *bio) +{ + struct request_queue *q = rq->q; + struct bfq_data *bfqd = q->elevator->elevator_data; + struct bfq_io_cq *bic; + const int is_sync = rq_is_sync(rq); + struct bfq_queue *bfqq; + bool bfqq_already_existing = false, split = false; + bool new_queue = false; + + if (!rq->elv.icq) + return; + bic = icq_to_bic(rq->elv.icq); + + spin_lock_irq(&bfqd->lock); + + bfq_check_ioprio_change(bic, bio); + + bfq_bic_update_cgroup(bic, bio); + + bfqq = bfq_get_bfqq_handle_split(bfqd, bic, bio, false, is_sync, + &new_queue); + + if (likely(!new_queue)) { + /* If the queue was seeky for too long, break it apart. */ + if (bfq_bfqq_coop(bfqq) && bfq_bfqq_split_coop(bfqq)) { + BUG_ON(!is_sync); + bfq_log_bfqq(bfqd, bfqq, "breaking apart bfqq"); + + /* Update bic before losing reference to bfqq */ + if (bfq_bfqq_in_large_burst(bfqq)) + bic->saved_in_large_burst = true; + + bfqq = bfq_split_bfqq(bic, bfqq); + + if (!bfqq) + bfqq = bfq_get_bfqq_handle_split(bfqd, bic, bio, + true, is_sync, + NULL); + else + bfqq_already_existing = true; + + BUG_ON(!bfqq); + BUG_ON(bfqq == &bfqd->oom_bfqq); + } + } + + bfqq->allocated++; + bfq_log_bfqq(bfqq->bfqd, bfqq, + "get_request: new allocated %d", bfqq->allocated); + + bfqq->ref++; + bfq_log_bfqq(bfqd, bfqq, "get_request %p: bfqq %p, %d", rq, bfqq, bfqq->ref); + + rq->elv.priv[0] = bic; + rq->elv.priv[1] = bfqq; + rq->rq_flags &= ~RQF_DISP_LIST; + + /* + * If a bfq_queue has only one process reference, it is owned + * by only this bic: we can then set bfqq->bic = bic. in + * addition, if the queue has also just been split, we have to + * resume its state. + */ + if (likely(bfqq != &bfqd->oom_bfqq) && bfqq_process_refs(bfqq) == 1) { + bfqq->bic = bic; + if (split) { + /* + * The queue has just been split from a shared + * queue: restore the idle window and the + * possible weight raising period. + */ + bfq_bfqq_resume_state(bfqq, bfqd, bic, + bfqq_already_existing); + } + } + + if (unlikely(bfq_bfqq_just_created(bfqq))) + bfq_handle_burst(bfqd, bfqq); + + rq->rq_flags |= RQF_GOT; + spin_unlock_irq(&bfqd->lock); +} + +static void bfq_idle_slice_timer_body(struct bfq_queue *bfqq) +{ + struct bfq_data *bfqd = bfqq->bfqd; + enum bfqq_expiration reason; + unsigned long flags; + + BUG_ON(!bfqd); + spin_lock_irqsave(&bfqd->lock, flags); + + bfq_log_bfqq(bfqd, bfqq, "handling slice_timer expiration"); + bfq_clear_bfqq_wait_request(bfqq); + + if (bfqq != bfqd->in_service_queue) { + spin_unlock_irqrestore(&bfqd->lock, flags); + return; + } + + if (bfq_bfqq_budget_timeout(bfqq)) + /* + * Also here the queue can be safely expired + * for budget timeout without wasting + * guarantees + */ + reason = BFQ_BFQQ_BUDGET_TIMEOUT; + else if (bfqq->queued[0] == 0 && bfqq->queued[1] == 0) + /* + * The queue may not be empty upon timer expiration, + * because we may not disable the timer when the + * first request of the in-service queue arrives + * during disk idling. + */ + reason = BFQ_BFQQ_TOO_IDLE; + else + goto schedule_dispatch; + + bfq_bfqq_expire(bfqd, bfqq, true, reason); + +schedule_dispatch: + spin_unlock_irqrestore(&bfqd->lock, flags); + bfq_schedule_dispatch(bfqd); +} + +/* + * Handler of the expiration of the timer running if the in-service queue + * is idling inside its time slice. + */ +static enum hrtimer_restart bfq_idle_slice_timer(struct hrtimer *timer) +{ + struct bfq_data *bfqd = container_of(timer, struct bfq_data, + idle_slice_timer); + struct bfq_queue *bfqq = bfqd->in_service_queue; + + bfq_log(bfqd, "slice_timer expired"); + + /* + * Theoretical race here: the in-service queue can be NULL or + * different from the queue that was idling if a new request + * arrives for the current queue and there is a full dispatch + * cycle that changes the in-service queue. This can hardly + * happen, but in the worst case we just expire a queue too + * early. + */ + if (bfqq) + bfq_idle_slice_timer_body(bfqq); + + return HRTIMER_NORESTART; +} + +static void __bfq_put_async_bfqq(struct bfq_data *bfqd, + struct bfq_queue **bfqq_ptr) +{ + struct bfq_group *root_group = bfqd->root_group; + struct bfq_queue *bfqq = *bfqq_ptr; + + bfq_log(bfqd, "put_async_bfqq: %p", bfqq); + if (bfqq) { + bfq_bfqq_move(bfqd, bfqq, root_group); + bfq_log_bfqq(bfqd, bfqq, "put_async_bfqq: putting %p, %d", + bfqq, bfqq->ref); + bfq_put_queue(bfqq); + *bfqq_ptr = NULL; + } +} + +/* + * Release all the bfqg references to its async queues. If we are + * deallocating the group these queues may still contain requests, so + * we reparent them to the root cgroup (i.e., the only one that will + * exist for sure until all the requests on a device are gone). + */ +static void bfq_put_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg) +{ + int i, j; + + for (i = 0; i < 2; i++) + for (j = 0; j < IOPRIO_BE_NR; j++) + __bfq_put_async_bfqq(bfqd, &bfqg->async_bfqq[i][j]); + + __bfq_put_async_bfqq(bfqd, &bfqg->async_idle_bfqq); +} + +static void bfq_exit_queue(struct elevator_queue *e) +{ + struct bfq_data *bfqd = e->elevator_data; + struct bfq_queue *bfqq, *n; + + bfq_log(bfqd, "exit_queue: starting ..."); + + hrtimer_cancel(&bfqd->idle_slice_timer); + + BUG_ON(bfqd->in_service_queue); + BUG_ON(!list_empty(&bfqd->active_list)); + + spin_lock_irq(&bfqd->lock); + list_for_each_entry_safe(bfqq, n, &bfqd->idle_list, bfqq_list) + bfq_deactivate_bfqq(bfqd, bfqq, false, false); + spin_unlock_irq(&bfqd->lock); + + hrtimer_cancel(&bfqd->idle_slice_timer); + + BUG_ON(hrtimer_active(&bfqd->idle_slice_timer)); + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + blkcg_deactivate_policy(bfqd->queue, &blkcg_policy_bfq); +#else + spin_lock_irq(&bfqd->lock); + bfq_put_async_queues(bfqd, bfqd->root_group); + kfree(bfqd->root_group); + spin_unlock_irq(&bfqd->lock); +#endif + + bfq_log(bfqd, "exit_queue: finished ..."); + kfree(bfqd); +} + +static void bfq_init_root_group(struct bfq_group *root_group, + struct bfq_data *bfqd) +{ + int i; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + root_group->entity.parent = NULL; + root_group->my_entity = NULL; + root_group->bfqd = bfqd; +#endif + root_group->rq_pos_tree = RB_ROOT; + for (i = 0; i < BFQ_IOPRIO_CLASSES; i++) + root_group->sched_data.service_tree[i] = BFQ_SERVICE_TREE_INIT; + root_group->sched_data.bfq_class_idle_last_service = jiffies; +} + +static int bfq_init_queue(struct request_queue *q, struct elevator_type *e) +{ + struct bfq_data *bfqd; + struct elevator_queue *eq; + + eq = elevator_alloc(q, e); + if (!eq) + return -ENOMEM; + + bfqd = kzalloc_node(sizeof(*bfqd), GFP_KERNEL, q->node); + if (!bfqd) { + kobject_put(&eq->kobj); + return -ENOMEM; + } + eq->elevator_data = bfqd; + + spin_lock_irq(q->queue_lock); + q->elevator = eq; + spin_unlock_irq(q->queue_lock); + + /* + * Our fallback bfqq if bfq_find_alloc_queue() runs into OOM issues. + * Grab a permanent reference to it, so that the normal code flow + * will not attempt to free it. + */ + bfq_init_bfqq(bfqd, &bfqd->oom_bfqq, NULL, 1, 0); + bfqd->oom_bfqq.ref++; + bfqd->oom_bfqq.new_ioprio = BFQ_DEFAULT_QUEUE_IOPRIO; + bfqd->oom_bfqq.new_ioprio_class = IOPRIO_CLASS_BE; + bfqd->oom_bfqq.entity.new_weight = + bfq_ioprio_to_weight(bfqd->oom_bfqq.new_ioprio); + + /* oom_bfqq does not participate to bursts */ + bfq_clear_bfqq_just_created(&bfqd->oom_bfqq); + /* + * Trigger weight initialization, according to ioprio, at the + * oom_bfqq's first activation. The oom_bfqq's ioprio and ioprio + * class won't be changed any more. + */ + bfqd->oom_bfqq.entity.prio_changed = 1; + + bfqd->queue = q; + INIT_LIST_HEAD(&bfqd->dispatch); + + hrtimer_init(&bfqd->idle_slice_timer, CLOCK_MONOTONIC, + HRTIMER_MODE_REL); + bfqd->idle_slice_timer.function = bfq_idle_slice_timer; + + bfqd->queue_weights_tree = RB_ROOT; + bfqd->group_weights_tree = RB_ROOT; + + INIT_LIST_HEAD(&bfqd->active_list); + INIT_LIST_HEAD(&bfqd->idle_list); + INIT_HLIST_HEAD(&bfqd->burst_list); + + bfqd->hw_tag = -1; + + bfqd->bfq_max_budget = bfq_default_max_budget; + + bfqd->bfq_fifo_expire[0] = bfq_fifo_expire[0]; + bfqd->bfq_fifo_expire[1] = bfq_fifo_expire[1]; + bfqd->bfq_back_max = bfq_back_max; + bfqd->bfq_back_penalty = bfq_back_penalty; + bfqd->bfq_slice_idle = bfq_slice_idle; + bfqd->bfq_timeout = bfq_timeout; + + bfqd->bfq_requests_within_timer = 120; + + bfqd->bfq_large_burst_thresh = 8; + bfqd->bfq_burst_interval = msecs_to_jiffies(180); + + bfqd->low_latency = true; + + /* + * Trade-off between responsiveness and fairness. + */ + bfqd->bfq_wr_coeff = 30; + bfqd->bfq_wr_rt_max_time = msecs_to_jiffies(300); + bfqd->bfq_wr_max_time = 0; + bfqd->bfq_wr_min_idle_time = msecs_to_jiffies(2000); + bfqd->bfq_wr_min_inter_arr_async = msecs_to_jiffies(500); + bfqd->bfq_wr_max_softrt_rate = 7000; /* + * Approximate rate required + * to playback or record a + * high-definition compressed + * video. + */ + bfqd->wr_busy_queues = 0; + + /* + * Begin by assuming, optimistically, that the device is a + * high-speed one, and that its peak rate is equal to 2/3 of + * the highest reference rate. + */ + bfqd->RT_prod = R_fast[blk_queue_nonrot(bfqd->queue)] * + T_fast[blk_queue_nonrot(bfqd->queue)]; + bfqd->peak_rate = R_fast[blk_queue_nonrot(bfqd->queue)] * 2 / 3; + bfqd->device_speed = BFQ_BFQD_FAST; + + spin_lock_init(&bfqd->lock); + + /* + * The invocation of the next bfq_create_group_hierarchy + * function is the head of a chain of function calls + * (bfq_create_group_hierarchy->blkcg_activate_policy-> + * blk_mq_freeze_queue) that may lead to the invocation of the + * has_work hook function. For this reason, + * bfq_create_group_hierarchy is invoked only after all + * scheduler data has been initialized, apart from the fields + * that can be initialized only after invoking + * bfq_create_group_hierarchy. This, in particular, enables + * has_work to correctly return false. Of course, to avoid + * other inconsistencies, the blk-mq stack must then refrain + * from invoking further scheduler hooks before this init + * function is finished. + */ + bfqd->root_group = bfq_create_group_hierarchy(bfqd, q->node); + if (!bfqd->root_group) + goto out_free; + bfq_init_root_group(bfqd->root_group, bfqd); + bfq_init_entity(&bfqd->oom_bfqq.entity, bfqd->root_group); + + return 0; + +out_free: + kfree(bfqd); + kobject_put(&eq->kobj); + return -ENOMEM; +} + +static void bfq_slab_kill(void) +{ + kmem_cache_destroy(bfq_pool); +} + +static int __init bfq_slab_setup(void) +{ + bfq_pool = KMEM_CACHE(bfq_queue, 0); + if (!bfq_pool) + return -ENOMEM; + return 0; +} + +static ssize_t bfq_var_show(unsigned int var, char *page) +{ + return sprintf(page, "%u\n", var); +} + +static ssize_t bfq_var_store(unsigned long *var, const char *page, + size_t count) +{ + unsigned long new_val; + int ret = kstrtoul(page, 10, &new_val); + + if (ret == 0) + *var = new_val; + + return count; +} + +static ssize_t bfq_wr_max_time_show(struct elevator_queue *e, char *page) +{ + struct bfq_data *bfqd = e->elevator_data; + + return sprintf(page, "%d\n", bfqd->bfq_wr_max_time > 0 ? + jiffies_to_msecs(bfqd->bfq_wr_max_time) : + jiffies_to_msecs(bfq_wr_duration(bfqd))); +} + +static ssize_t bfq_weights_show(struct elevator_queue *e, char *page) +{ + struct bfq_queue *bfqq; + struct bfq_data *bfqd = e->elevator_data; + ssize_t num_char = 0; + + num_char += sprintf(page + num_char, "Tot reqs queued %d\n\n", + bfqd->queued); + + spin_lock_irq(&bfqd->lock); + + num_char += sprintf(page + num_char, "Active:\n"); + list_for_each_entry(bfqq, &bfqd->active_list, bfqq_list) { + num_char += sprintf(page + num_char, + "pid%d: weight %hu, nr_queued %d %d, ", + bfqq->pid, + bfqq->entity.weight, + bfqq->queued[0], + bfqq->queued[1]); + num_char += sprintf(page + num_char, + "dur %d/%u\n", + jiffies_to_msecs( + jiffies - + bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } + + num_char += sprintf(page + num_char, "Idle:\n"); + list_for_each_entry(bfqq, &bfqd->idle_list, bfqq_list) { + num_char += sprintf(page + num_char, + "pid%d: weight %hu, dur %d/%u\n", + bfqq->pid, + bfqq->entity.weight, + jiffies_to_msecs(jiffies - + bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } + + spin_unlock_irq(&bfqd->lock); + + return num_char; +} + +#define SHOW_FUNCTION(__FUNC, __VAR, __CONV) \ +static ssize_t __FUNC(struct elevator_queue *e, char *page) \ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + u64 __data = __VAR; \ + if (__CONV == 1) \ + __data = jiffies_to_msecs(__data); \ + else if (__CONV == 2) \ + __data = div_u64(__data, NSEC_PER_MSEC); \ + return bfq_var_show(__data, (page)); \ +} +SHOW_FUNCTION(bfq_fifo_expire_sync_show, bfqd->bfq_fifo_expire[1], 2); +SHOW_FUNCTION(bfq_fifo_expire_async_show, bfqd->bfq_fifo_expire[0], 2); +SHOW_FUNCTION(bfq_back_seek_max_show, bfqd->bfq_back_max, 0); +SHOW_FUNCTION(bfq_back_seek_penalty_show, bfqd->bfq_back_penalty, 0); +SHOW_FUNCTION(bfq_slice_idle_show, bfqd->bfq_slice_idle, 2); +SHOW_FUNCTION(bfq_max_budget_show, bfqd->bfq_user_max_budget, 0); +SHOW_FUNCTION(bfq_timeout_sync_show, bfqd->bfq_timeout, 1); +SHOW_FUNCTION(bfq_strict_guarantees_show, bfqd->strict_guarantees, 0); +SHOW_FUNCTION(bfq_low_latency_show, bfqd->low_latency, 0); +SHOW_FUNCTION(bfq_wr_coeff_show, bfqd->bfq_wr_coeff, 0); +SHOW_FUNCTION(bfq_wr_rt_max_time_show, bfqd->bfq_wr_rt_max_time, 1); +SHOW_FUNCTION(bfq_wr_min_idle_time_show, bfqd->bfq_wr_min_idle_time, 1); +SHOW_FUNCTION(bfq_wr_min_inter_arr_async_show, bfqd->bfq_wr_min_inter_arr_async, + 1); +SHOW_FUNCTION(bfq_wr_max_softrt_rate_show, bfqd->bfq_wr_max_softrt_rate, 0); +#undef SHOW_FUNCTION + +#define USEC_SHOW_FUNCTION(__FUNC, __VAR) \ +static ssize_t __FUNC(struct elevator_queue *e, char *page) \ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + u64 __data = __VAR; \ + __data = div_u64(__data, NSEC_PER_USEC); \ + return bfq_var_show(__data, (page)); \ +} +USEC_SHOW_FUNCTION(bfq_slice_idle_us_show, bfqd->bfq_slice_idle); +#undef USEC_SHOW_FUNCTION + +#define STORE_FUNCTION(__FUNC, __PTR, MIN, MAX, __CONV) \ +static ssize_t \ +__FUNC(struct elevator_queue *e, const char *page, size_t count) \ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + unsigned long uninitialized_var(__data); \ + int ret = bfq_var_store(&__data, (page), count); \ + if (__data < (MIN)) \ + __data = (MIN); \ + else if (__data > (MAX)) \ + __data = (MAX); \ + if (__CONV == 1) \ + *(__PTR) = msecs_to_jiffies(__data); \ + else if (__CONV == 2) \ + *(__PTR) = (u64)__data * NSEC_PER_MSEC; \ + else \ + *(__PTR) = __data; \ + return ret; \ +} +STORE_FUNCTION(bfq_fifo_expire_sync_store, &bfqd->bfq_fifo_expire[1], 1, + INT_MAX, 2); +STORE_FUNCTION(bfq_fifo_expire_async_store, &bfqd->bfq_fifo_expire[0], 1, + INT_MAX, 2); +STORE_FUNCTION(bfq_back_seek_max_store, &bfqd->bfq_back_max, 0, INT_MAX, 0); +STORE_FUNCTION(bfq_back_seek_penalty_store, &bfqd->bfq_back_penalty, 1, + INT_MAX, 0); +STORE_FUNCTION(bfq_slice_idle_store, &bfqd->bfq_slice_idle, 0, INT_MAX, 2); +STORE_FUNCTION(bfq_wr_coeff_store, &bfqd->bfq_wr_coeff, 1, INT_MAX, 0); +STORE_FUNCTION(bfq_wr_max_time_store, &bfqd->bfq_wr_max_time, 0, INT_MAX, 1); +STORE_FUNCTION(bfq_wr_rt_max_time_store, &bfqd->bfq_wr_rt_max_time, 0, INT_MAX, + 1); +STORE_FUNCTION(bfq_wr_min_idle_time_store, &bfqd->bfq_wr_min_idle_time, 0, + INT_MAX, 1); +STORE_FUNCTION(bfq_wr_min_inter_arr_async_store, + &bfqd->bfq_wr_min_inter_arr_async, 0, INT_MAX, 1); +STORE_FUNCTION(bfq_wr_max_softrt_rate_store, &bfqd->bfq_wr_max_softrt_rate, 0, + INT_MAX, 0); +#undef STORE_FUNCTION + +#define USEC_STORE_FUNCTION(__FUNC, __PTR, MIN, MAX) \ +static ssize_t __FUNC(struct elevator_queue *e, const char *page, size_t count)\ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + unsigned long uninitialized_var(__data); \ + int ret = bfq_var_store(&__data, (page), count); \ + if (__data < (MIN)) \ + __data = (MIN); \ + else if (__data > (MAX)) \ + __data = (MAX); \ + *(__PTR) = (u64)__data * NSEC_PER_USEC; \ + return ret; \ +} +USEC_STORE_FUNCTION(bfq_slice_idle_us_store, &bfqd->bfq_slice_idle, 0, + UINT_MAX); +#undef USEC_STORE_FUNCTION + +/* do nothing for the moment */ +static ssize_t bfq_weights_store(struct elevator_queue *e, + const char *page, size_t count) +{ + return count; +} + +static ssize_t bfq_max_budget_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data == 0) + bfqd->bfq_max_budget = bfq_calc_max_budget(bfqd); + else { + if (__data > INT_MAX) + __data = INT_MAX; + bfqd->bfq_max_budget = __data; + } + + bfqd->bfq_user_max_budget = __data; + + return ret; +} + +/* + * Leaving this name to preserve name compatibility with cfq + * parameters, but this timeout is used for both sync and async. + */ +static ssize_t bfq_timeout_sync_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data < 1) + __data = 1; + else if (__data > INT_MAX) + __data = INT_MAX; + + bfqd->bfq_timeout = msecs_to_jiffies(__data); + if (bfqd->bfq_user_max_budget == 0) + bfqd->bfq_max_budget = bfq_calc_max_budget(bfqd); + + return ret; +} + +static ssize_t bfq_strict_guarantees_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data > 1) + __data = 1; + if (!bfqd->strict_guarantees && __data == 1 + && bfqd->bfq_slice_idle < 8 * NSEC_PER_MSEC) + bfqd->bfq_slice_idle = 8 * NSEC_PER_MSEC; + + bfqd->strict_guarantees = __data; + + return ret; +} + +static ssize_t bfq_low_latency_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data > 1) + __data = 1; + if (__data == 0 && bfqd->low_latency != 0) + bfq_end_wr(bfqd); + bfqd->low_latency = __data; + + return ret; +} + +#define BFQ_ATTR(name) \ + __ATTR(name, S_IRUGO|S_IWUSR, bfq_##name##_show, bfq_##name##_store) + +static struct elv_fs_entry bfq_attrs[] = { + BFQ_ATTR(fifo_expire_sync), + BFQ_ATTR(fifo_expire_async), + BFQ_ATTR(back_seek_max), + BFQ_ATTR(back_seek_penalty), + BFQ_ATTR(slice_idle), + BFQ_ATTR(slice_idle_us), + BFQ_ATTR(max_budget), + BFQ_ATTR(timeout_sync), + BFQ_ATTR(strict_guarantees), + BFQ_ATTR(low_latency), + BFQ_ATTR(wr_coeff), + BFQ_ATTR(wr_max_time), + BFQ_ATTR(wr_rt_max_time), + BFQ_ATTR(wr_min_idle_time), + BFQ_ATTR(wr_min_inter_arr_async), + BFQ_ATTR(wr_max_softrt_rate), + BFQ_ATTR(weights), + __ATTR_NULL +}; + +static struct elevator_type iosched_bfq_mq = { + .ops.mq = { + .prepare_request = bfq_prepare_request, + .finish_request = bfq_finish_request, + .exit_icq = bfq_exit_icq, + .insert_requests = bfq_insert_requests, + .dispatch_request = bfq_dispatch_request, + .next_request = elv_rb_latter_request, + .former_request = elv_rb_former_request, + .allow_merge = bfq_allow_bio_merge, + .bio_merge = bfq_bio_merge, + .request_merge = bfq_request_merge, + .requests_merged = bfq_requests_merged, + .request_merged = bfq_request_merged, + .has_work = bfq_has_work, + .init_sched = bfq_init_queue, + .exit_sched = bfq_exit_queue, + }, + + .uses_mq = true, + .icq_size = sizeof(struct bfq_io_cq), + .icq_align = __alignof__(struct bfq_io_cq), + .elevator_attrs = bfq_attrs, + .elevator_name = "bfq-mq", + .elevator_owner = THIS_MODULE, +}; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static struct blkcg_policy blkcg_policy_bfq = { + .dfl_cftypes = bfq_blkg_files, + .legacy_cftypes = bfq_blkcg_legacy_files, + + .cpd_alloc_fn = bfq_cpd_alloc, + .cpd_init_fn = bfq_cpd_init, + .cpd_bind_fn = bfq_cpd_init, + .cpd_free_fn = bfq_cpd_free, + + .pd_alloc_fn = bfq_pd_alloc, + .pd_init_fn = bfq_pd_init, + .pd_offline_fn = bfq_pd_offline, + .pd_free_fn = bfq_pd_free, + .pd_reset_stats_fn = bfq_pd_reset_stats, +}; +#endif + +static int __init bfq_init(void) +{ + int ret; + char msg[60] = "BFQ I/O-scheduler: v8r12"; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + ret = blkcg_policy_register(&blkcg_policy_bfq); + if (ret) + return ret; +#endif + + ret = -ENOMEM; + if (bfq_slab_setup()) + goto err_pol_unreg; + + /* + * Times to load large popular applications for the typical + * systems installed on the reference devices (see the + * comments before the definitions of the next two + * arrays). Actually, we use slightly slower values, as the + * estimated peak rate tends to be smaller than the actual + * peak rate. The reason for this last fact is that estimates + * are computed over much shorter time intervals than the long + * intervals typically used for benchmarking. Why? First, to + * adapt more quickly to variations. Second, because an I/O + * scheduler cannot rely on a peak-rate-evaluation workload to + * be run for a long time. + */ + T_slow[0] = msecs_to_jiffies(3500); /* actually 4 sec */ + T_slow[1] = msecs_to_jiffies(6000); /* actually 6.5 sec */ + T_fast[0] = msecs_to_jiffies(7000); /* actually 8 sec */ + T_fast[1] = msecs_to_jiffies(2500); /* actually 3 sec */ + + /* + * Thresholds that determine the switch between speed classes + * (see the comments before the definition of the array + * device_speed_thresh). These thresholds are biased towards + * transitions to the fast class. This is safer than the + * opposite bias. In fact, a wrong transition to the slow + * class results in short weight-raising periods, because the + * speed of the device then tends to be higher that the + * reference peak rate. On the opposite end, a wrong + * transition to the fast class tends to increase + * weight-raising periods, because of the opposite reason. + */ + device_speed_thresh[0] = (4 * R_slow[0]) / 3; + device_speed_thresh[1] = (4 * R_slow[1]) / 3; + + ret = elv_register(&iosched_bfq_mq); + if (ret) + goto err_pol_unreg; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + strcat(msg, " (with cgroups support)"); +#endif + pr_info("%s", msg); + + return 0; + +err_pol_unreg: +#ifdef BFQ_GROUP_IOSCHED_ENABLED + blkcg_policy_unregister(&blkcg_policy_bfq); +#endif + return ret; +} + +static void __exit bfq_exit(void) +{ + elv_unregister(&iosched_bfq_mq); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + blkcg_policy_unregister(&blkcg_policy_bfq); +#endif + bfq_slab_kill(); +} + +module_init(bfq_init); +module_exit(bfq_exit); + +MODULE_AUTHOR("Paolo Valente"); +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("MQ Budget Fair Queueing I/O Scheduler"); diff --git b/block/bfq-mq.h b/block/bfq-mq.h new file mode 100644 index 0000000..7ed2cc2 --- /dev/null +++ b/block/bfq-mq.h @@ -0,0 +1,987 @@ +/* + * BFQ v8r12 for 4.11.0: data structures and common functions prototypes. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2015 Paolo Valente + * + * Copyright (C) 2017 Paolo Valente + */ + +#ifndef _BFQ_H +#define _BFQ_H + +#include +#include + +/* see comments on CONFIG_BFQ_GROUP_IOSCHED in bfq.h */ +#ifdef CONFIG_MQ_BFQ_GROUP_IOSCHED +#define BFQ_GROUP_IOSCHED_ENABLED +#endif + +#define BFQ_IOPRIO_CLASSES 3 +#define BFQ_CL_IDLE_TIMEOUT (HZ/5) + +#define BFQ_MIN_WEIGHT 1 +#define BFQ_MAX_WEIGHT 1000 +#define BFQ_WEIGHT_CONVERSION_COEFF 10 + +#define BFQ_DEFAULT_QUEUE_IOPRIO 4 + +#define BFQ_WEIGHT_LEGACY_DFL 100 +#define BFQ_DEFAULT_GRP_IOPRIO 0 +#define BFQ_DEFAULT_GRP_CLASS IOPRIO_CLASS_BE + +/* + * Soft real-time applications are extremely more latency sensitive + * than interactive ones. Over-raise the weight of the former to + * privilege them against the latter. + */ +#define BFQ_SOFTRT_WEIGHT_FACTOR 100 + +struct bfq_entity; + +/** + * struct bfq_service_tree - per ioprio_class service tree. + * + * Each service tree represents a B-WF2Q+ scheduler on its own. Each + * ioprio_class has its own independent scheduler, and so its own + * bfq_service_tree. All the fields are protected by the queue lock + * of the containing bfqd. + */ +struct bfq_service_tree { + /* tree for active entities (i.e., those backlogged) */ + struct rb_root active; + /* tree for idle entities (i.e., not backlogged, with V <= F_i)*/ + struct rb_root idle; + + struct bfq_entity *first_idle; /* idle entity with minimum F_i */ + struct bfq_entity *last_idle; /* idle entity with maximum F_i */ + + u64 vtime; /* scheduler virtual time */ + /* scheduler weight sum; active and idle entities contribute to it */ + unsigned long wsum; +}; + +/** + * struct bfq_sched_data - multi-class scheduler. + * + * bfq_sched_data is the basic scheduler queue. It supports three + * ioprio_classes, and can be used either as a toplevel queue or as an + * intermediate queue in a hierarchical setup. + * + * The supported ioprio_classes are the same as in CFQ, in descending + * priority order, IOPRIO_CLASS_RT, IOPRIO_CLASS_BE, IOPRIO_CLASS_IDLE. + * Requests from higher priority queues are served before all the + * requests from lower priority queues; among requests of the same + * queue requests are served according to B-WF2Q+. + * + * The schedule is implemented by the service trees, plus the field + * @next_in_service, which points to the entity on the active trees + * that will be served next, if 1) no changes in the schedule occurs + * before the current in-service entity is expired, 2) the in-service + * queue becomes idle when it expires, and 3) if the entity pointed by + * in_service_entity is not a queue, then the in-service child entity + * of the entity pointed by in_service_entity becomes idle on + * expiration. This peculiar definition allows for the following + * optimization, not yet exploited: while a given entity is still in + * service, we already know which is the best candidate for next + * service among the other active entitities in the same parent + * entity. We can then quickly compare the timestamps of the + * in-service entity with those of such best candidate. + * + * All the fields are protected by the queue lock of the containing + * bfqd. + */ +struct bfq_sched_data { + struct bfq_entity *in_service_entity; /* entity in service */ + /* head-of-the-line entity in the scheduler (see comments above) */ + struct bfq_entity *next_in_service; + /* array of service trees, one per ioprio_class */ + struct bfq_service_tree service_tree[BFQ_IOPRIO_CLASSES]; + /* last time CLASS_IDLE was served */ + unsigned long bfq_class_idle_last_service; + +}; + +/** + * struct bfq_weight_counter - counter of the number of all active entities + * with a given weight. + */ +struct bfq_weight_counter { + unsigned int weight; /* weight of the entities this counter refers to */ + unsigned int num_active; /* nr of active entities with this weight */ + /* + * Weights tree member (see bfq_data's @queue_weights_tree and + * @group_weights_tree) + */ + struct rb_node weights_node; +}; + +/** + * struct bfq_entity - schedulable entity. + * + * A bfq_entity is used to represent either a bfq_queue (leaf node in the + * cgroup hierarchy) or a bfq_group into the upper level scheduler. Each + * entity belongs to the sched_data of the parent group in the cgroup + * hierarchy. Non-leaf entities have also their own sched_data, stored + * in @my_sched_data. + * + * Each entity stores independently its priority values; this would + * allow different weights on different devices, but this + * functionality is not exported to userspace by now. Priorities and + * weights are updated lazily, first storing the new values into the + * new_* fields, then setting the @prio_changed flag. As soon as + * there is a transition in the entity state that allows the priority + * update to take place the effective and the requested priority + * values are synchronized. + * + * Unless cgroups are used, the weight value is calculated from the + * ioprio to export the same interface as CFQ. When dealing with + * ``well-behaved'' queues (i.e., queues that do not spend too much + * time to consume their budget and have true sequential behavior, and + * when there are no external factors breaking anticipation) the + * relative weights at each level of the cgroups hierarchy should be + * guaranteed. All the fields are protected by the queue lock of the + * containing bfqd. + */ +struct bfq_entity { + struct rb_node rb_node; /* service_tree member */ + /* pointer to the weight counter associated with this entity */ + struct bfq_weight_counter *weight_counter; + + /* + * Flag, true if the entity is on a tree (either the active or + * the idle one of its service_tree) or is in service. + */ + bool on_st; + + u64 finish; /* B-WF2Q+ finish timestamp (aka F_i) */ + u64 start; /* B-WF2Q+ start timestamp (aka S_i) */ + + /* tree the entity is enqueued into; %NULL if not on a tree */ + struct rb_root *tree; + + /* + * minimum start time of the (active) subtree rooted at this + * entity; used for O(log N) lookups into active trees + */ + u64 min_start; + + /* amount of service received during the last service slot */ + int service; + + /* budget, used also to calculate F_i: F_i = S_i + @budget / @weight */ + int budget; + + unsigned int weight; /* weight of the queue */ + unsigned int new_weight; /* next weight if a change is in progress */ + + /* original weight, used to implement weight boosting */ + unsigned int orig_weight; + + /* parent entity, for hierarchical scheduling */ + struct bfq_entity *parent; + + /* + * For non-leaf nodes in the hierarchy, the associated + * scheduler queue, %NULL on leaf nodes. + */ + struct bfq_sched_data *my_sched_data; + /* the scheduler queue this entity belongs to */ + struct bfq_sched_data *sched_data; + + /* flag, set to request a weight, ioprio or ioprio_class change */ + int prio_changed; +}; + +struct bfq_group; + +/** + * struct bfq_ttime - per process thinktime stats. + */ +struct bfq_ttime { + u64 last_end_request; /* completion time of last request */ + + u64 ttime_total; /* total process thinktime */ + unsigned long ttime_samples; /* number of thinktime samples */ + u64 ttime_mean; /* average process thinktime */ + +}; + +/** + * struct bfq_queue - leaf schedulable entity. + * + * A bfq_queue is a leaf request queue; it can be associated with an + * io_context or more, if it is async or shared between cooperating + * processes. @cgroup holds a reference to the cgroup, to be sure that it + * does not disappear while a bfqq still references it (mostly to avoid + * races between request issuing and task migration followed by cgroup + * destruction). + * All the fields are protected by the queue lock of the containing bfqd. + */ +struct bfq_queue { + /* reference counter */ + int ref; + /* parent bfq_data */ + struct bfq_data *bfqd; + + /* current ioprio and ioprio class */ + unsigned short ioprio, ioprio_class; + /* next ioprio and ioprio class if a change is in progress */ + unsigned short new_ioprio, new_ioprio_class; + + /* + * Shared bfq_queue if queue is cooperating with one or more + * other queues. + */ + struct bfq_queue *new_bfqq; + /* request-position tree member (see bfq_group's @rq_pos_tree) */ + struct rb_node pos_node; + /* request-position tree root (see bfq_group's @rq_pos_tree) */ + struct rb_root *pos_root; + + /* sorted list of pending requests */ + struct rb_root sort_list; + /* if fifo isn't expired, next request to serve */ + struct request *next_rq; + /* number of sync and async requests queued */ + int queued[2]; + /* number of requests currently allocated */ + int allocated; + /* number of pending metadata requests */ + int meta_pending; + /* fifo list of requests in sort_list */ + struct list_head fifo; + + /* entity representing this queue in the scheduler */ + struct bfq_entity entity; + + /* maximum budget allowed from the feedback mechanism */ + int max_budget; + /* budget expiration (in jiffies) */ + unsigned long budget_timeout; + + /* number of requests on the dispatch list or inside driver */ + int dispatched; + + unsigned int flags; /* status flags.*/ + + /* node for active/idle bfqq list inside parent bfqd */ + struct list_head bfqq_list; + + /* associated @bfq_ttime struct */ + struct bfq_ttime ttime; + + /* bit vector: a 1 for each seeky requests in history */ + u32 seek_history; + + /* node for the device's burst list */ + struct hlist_node burst_list_node; + + /* position of the last request enqueued */ + sector_t last_request_pos; + + /* Number of consecutive pairs of request completion and + * arrival, such that the queue becomes idle after the + * completion, but the next request arrives within an idle + * time slice; used only if the queue's IO_bound flag has been + * cleared. + */ + unsigned int requests_within_timer; + + /* pid of the process owning the queue, used for logging purposes */ + pid_t pid; + + /* + * Pointer to the bfq_io_cq owning the bfq_queue, set to %NULL + * if the queue is shared. + */ + struct bfq_io_cq *bic; + + /* current maximum weight-raising time for this queue */ + unsigned long wr_cur_max_time; + /* + * Minimum time instant such that, only if a new request is + * enqueued after this time instant in an idle @bfq_queue with + * no outstanding requests, then the task associated with the + * queue it is deemed as soft real-time (see the comments on + * the function bfq_bfqq_softrt_next_start()) + */ + unsigned long soft_rt_next_start; + /* + * Start time of the current weight-raising period if + * the @bfq-queue is being weight-raised, otherwise + * finish time of the last weight-raising period. + */ + unsigned long last_wr_start_finish; + /* factor by which the weight of this queue is multiplied */ + unsigned int wr_coeff; + /* + * Time of the last transition of the @bfq_queue from idle to + * backlogged. + */ + unsigned long last_idle_bklogged; + /* + * Cumulative service received from the @bfq_queue since the + * last transition from idle to backlogged. + */ + unsigned long service_from_backlogged; + /* + * Value of wr start time when switching to soft rt + */ + unsigned long wr_start_at_switch_to_srt; + + unsigned long split_time; /* time of last split */ +}; + +/** + * struct bfq_io_cq - per (request_queue, io_context) structure. + */ +struct bfq_io_cq { + /* associated io_cq structure */ + struct io_cq icq; /* must be the first member */ + /* array of two process queues, the sync and the async */ + struct bfq_queue *bfqq[2]; + /* per (request_queue, blkcg) ioprio */ + int ioprio; +#ifdef BFQ_GROUP_IOSCHED_ENABLED + uint64_t blkcg_serial_nr; /* the current blkcg serial */ +#endif + + /* + * Snapshot of the has_short_time flag before merging; taken + * to remember its value while the queue is merged, so as to + * be able to restore it in case of split. + */ + bool saved_has_short_ttime; + /* + * Same purpose as the previous two fields for the I/O bound + * classification of a queue. + */ + bool saved_IO_bound; + + /* + * Same purpose as the previous fields for the value of the + * field keeping the queue's belonging to a large burst + */ + bool saved_in_large_burst; + /* + * True if the queue belonged to a burst list before its merge + * with another cooperating queue. + */ + bool was_in_burst_list; + + /* + * Similar to previous fields: save wr information. + */ + unsigned long saved_wr_coeff; + unsigned long saved_last_wr_start_finish; + unsigned long saved_wr_start_at_switch_to_srt; + unsigned int saved_wr_cur_max_time; + struct bfq_ttime saved_ttime; +}; + +enum bfq_device_speed { + BFQ_BFQD_FAST, + BFQ_BFQD_SLOW, +}; + +/** + * struct bfq_data - per-device data structure. + * + * All the fields are protected by @lock. + */ +struct bfq_data { + /* device request queue */ + struct request_queue *queue; + /* dispatch queue */ + struct list_head dispatch; + + /* root bfq_group for the device */ + struct bfq_group *root_group; + + /* + * rbtree of weight counters of @bfq_queues, sorted by + * weight. Used to keep track of whether all @bfq_queues have + * the same weight. The tree contains one counter for each + * distinct weight associated to some active and not + * weight-raised @bfq_queue (see the comments to the functions + * bfq_weights_tree_[add|remove] for further details). + */ + struct rb_root queue_weights_tree; + /* + * rbtree of non-queue @bfq_entity weight counters, sorted by + * weight. Used to keep track of whether all @bfq_groups have + * the same weight. The tree contains one counter for each + * distinct weight associated to some active @bfq_group (see + * the comments to the functions bfq_weights_tree_[add|remove] + * for further details). + */ + struct rb_root group_weights_tree; + + /* + * Number of bfq_queues containing requests (including the + * queue in service, even if it is idling). + */ + int busy_queues; + /* number of weight-raised busy @bfq_queues */ + int wr_busy_queues; + /* number of queued requests */ + int queued; + /* number of requests dispatched and waiting for completion */ + int rq_in_driver; + + /* + * Maximum number of requests in driver in the last + * @hw_tag_samples completed requests. + */ + int max_rq_in_driver; + /* number of samples used to calculate hw_tag */ + int hw_tag_samples; + /* flag set to one if the driver is showing a queueing behavior */ + int hw_tag; + + /* number of budgets assigned */ + int budgets_assigned; + + /* + * Timer set when idling (waiting) for the next request from + * the queue in service. + */ + struct hrtimer idle_slice_timer; + + /* bfq_queue in service */ + struct bfq_queue *in_service_queue; + + /* on-disk position of the last served request */ + sector_t last_position; + + /* time of last request completion (ns) */ + u64 last_completion; + + /* time of first rq dispatch in current observation interval (ns) */ + u64 first_dispatch; + /* time of last rq dispatch in current observation interval (ns) */ + u64 last_dispatch; + + /* beginning of the last budget */ + ktime_t last_budget_start; + /* beginning of the last idle slice */ + ktime_t last_idling_start; + + /* number of samples in current observation interval */ + int peak_rate_samples; + /* num of samples of seq dispatches in current observation interval */ + u32 sequential_samples; + /* total num of sectors transferred in current observation interval */ + u64 tot_sectors_dispatched; + /* max rq size seen during current observation interval (sectors) */ + u32 last_rq_max_size; + /* time elapsed from first dispatch in current observ. interval (us) */ + u64 delta_from_first; + /* current estimate of device peak rate */ + u32 peak_rate; + + /* maximum budget allotted to a bfq_queue before rescheduling */ + int bfq_max_budget; + + /* list of all the bfq_queues active on the device */ + struct list_head active_list; + /* list of all the bfq_queues idle on the device */ + struct list_head idle_list; + + /* + * Timeout for async/sync requests; when it fires, requests + * are served in fifo order. + */ + u64 bfq_fifo_expire[2]; + /* weight of backward seeks wrt forward ones */ + unsigned int bfq_back_penalty; + /* maximum allowed backward seek */ + unsigned int bfq_back_max; + /* maximum idling time */ + u32 bfq_slice_idle; + + /* user-configured max budget value (0 for auto-tuning) */ + int bfq_user_max_budget; + /* + * Timeout for bfq_queues to consume their budget; used to + * prevent seeky queues from imposing long latencies to + * sequential or quasi-sequential ones (this also implies that + * seeky queues cannot receive guarantees in the service + * domain; after a timeout they are charged for the time they + * have been in service, to preserve fairness among them, but + * without service-domain guarantees). + */ + unsigned int bfq_timeout; + + /* + * Number of consecutive requests that must be issued within + * the idle time slice to set again idling to a queue which + * was marked as non-I/O-bound (see the definition of the + * IO_bound flag for further details). + */ + unsigned int bfq_requests_within_timer; + + /* + * Force device idling whenever needed to provide accurate + * service guarantees, without caring about throughput + * issues. CAVEAT: this may even increase latencies, in case + * of useless idling for processes that did stop doing I/O. + */ + bool strict_guarantees; + + /* + * Last time at which a queue entered the current burst of + * queues being activated shortly after each other; for more + * details about this and the following parameters related to + * a burst of activations, see the comments on the function + * bfq_handle_burst. + */ + unsigned long last_ins_in_burst; + /* + * Reference time interval used to decide whether a queue has + * been activated shortly after @last_ins_in_burst. + */ + unsigned long bfq_burst_interval; + /* number of queues in the current burst of queue activations */ + int burst_size; + + /* common parent entity for the queues in the burst */ + struct bfq_entity *burst_parent_entity; + /* Maximum burst size above which the current queue-activation + * burst is deemed as 'large'. + */ + unsigned long bfq_large_burst_thresh; + /* true if a large queue-activation burst is in progress */ + bool large_burst; + /* + * Head of the burst list (as for the above fields, more + * details in the comments on the function bfq_handle_burst). + */ + struct hlist_head burst_list; + + /* if set to true, low-latency heuristics are enabled */ + bool low_latency; + /* + * Maximum factor by which the weight of a weight-raised queue + * is multiplied. + */ + unsigned int bfq_wr_coeff; + /* maximum duration of a weight-raising period (jiffies) */ + unsigned int bfq_wr_max_time; + + /* Maximum weight-raising duration for soft real-time processes */ + unsigned int bfq_wr_rt_max_time; + /* + * Minimum idle period after which weight-raising may be + * reactivated for a queue (in jiffies). + */ + unsigned int bfq_wr_min_idle_time; + /* + * Minimum period between request arrivals after which + * weight-raising may be reactivated for an already busy async + * queue (in jiffies). + */ + unsigned long bfq_wr_min_inter_arr_async; + + /* Max service-rate for a soft real-time queue, in sectors/sec */ + unsigned int bfq_wr_max_softrt_rate; + /* + * Cached value of the product R*T, used for computing the + * maximum duration of weight raising automatically. + */ + u64 RT_prod; + /* device-speed class for the low-latency heuristic */ + enum bfq_device_speed device_speed; + + /* fallback dummy bfqq for extreme OOM conditions */ + struct bfq_queue oom_bfqq; + + spinlock_t lock; + + /* + * bic associated with the task issuing current bio for + * merging. This and the next field are used as a support to + * be able to perform the bic lookup, needed by bio-merge + * functions, before the scheduler lock is taken, and thus + * avoid taking the request-queue lock while the scheduler + * lock is being held. + */ + struct bfq_io_cq *bio_bic; + /* bfqq associated with the task issuing current bio for merging */ + struct bfq_queue *bio_bfqq; + /* Extra flag used only for TESTING */ + bool bio_bfqq_set; +}; + +enum bfqq_state_flags { + BFQ_BFQQ_FLAG_just_created = 0, /* queue just allocated */ + BFQ_BFQQ_FLAG_busy, /* has requests or is in service */ + BFQ_BFQQ_FLAG_wait_request, /* waiting for a request */ + BFQ_BFQQ_FLAG_non_blocking_wait_rq, /* + * waiting for a request + * without idling the device + */ + BFQ_BFQQ_FLAG_fifo_expire, /* FIFO checked in this slice */ + BFQ_BFQQ_FLAG_has_short_ttime, /* queue has a short think time */ + BFQ_BFQQ_FLAG_sync, /* synchronous queue */ + BFQ_BFQQ_FLAG_IO_bound, /* + * bfqq has timed-out at least once + * having consumed at most 2/10 of + * its budget + */ + BFQ_BFQQ_FLAG_in_large_burst, /* + * bfqq activated in a large burst, + * see comments to bfq_handle_burst. + */ + BFQ_BFQQ_FLAG_softrt_update, /* + * may need softrt-next-start + * update + */ + BFQ_BFQQ_FLAG_coop, /* bfqq is shared */ + BFQ_BFQQ_FLAG_split_coop /* shared bfqq will be split */ +}; + +#define BFQ_BFQQ_FNS(name) \ +static void bfq_mark_bfqq_##name(struct bfq_queue *bfqq) \ +{ \ + (bfqq)->flags |= (1 << BFQ_BFQQ_FLAG_##name); \ +} \ +static void bfq_clear_bfqq_##name(struct bfq_queue *bfqq) \ +{ \ + (bfqq)->flags &= ~(1 << BFQ_BFQQ_FLAG_##name); \ +} \ +static int bfq_bfqq_##name(const struct bfq_queue *bfqq) \ +{ \ + return ((bfqq)->flags & (1 << BFQ_BFQQ_FLAG_##name)) != 0; \ +} + +BFQ_BFQQ_FNS(just_created); +BFQ_BFQQ_FNS(busy); +BFQ_BFQQ_FNS(wait_request); +BFQ_BFQQ_FNS(non_blocking_wait_rq); +BFQ_BFQQ_FNS(fifo_expire); +BFQ_BFQQ_FNS(has_short_ttime); +BFQ_BFQQ_FNS(sync); +BFQ_BFQQ_FNS(IO_bound); +BFQ_BFQQ_FNS(in_large_burst); +BFQ_BFQQ_FNS(coop); +BFQ_BFQQ_FNS(split_coop); +BFQ_BFQQ_FNS(softrt_update); +#undef BFQ_BFQQ_FNS + +/* Logging facilities. */ +#ifdef CONFIG_BFQ_REDIRECT_TO_CONSOLE + +static const char *checked_dev_name(const struct device *dev) +{ + static const char nodev[] = "nodev"; + + if (dev) + return dev_name(dev); + + return nodev; +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq); +static struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) do { \ + pr_crit("%s bfq%d%c %s " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + (bfqq)->pid, \ + bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + bfqq_group(bfqq)->blkg_path, ##args); \ +} while (0) + +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do { \ + pr_crit("%s %s " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + bfqg->blkg_path, ##args); \ +} while (0) + +#else /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) \ + pr_crit("%s bfq%d%c " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + (bfqq)->pid, bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + ##args) +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do {} while (0) + +#endif /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log(bfqd, fmt, args...) \ + pr_crit("%s bfq " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + ##args) + +#else /* CONFIG_BFQ_REDIRECT_TO_CONSOLE */ + +#if !defined(CONFIG_BLK_DEV_IO_TRACE) + +/* Avoid possible "unused-variable" warning. See commit message. */ + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) ((void) (bfqq)) + +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) ((void) (bfqg)) + +#define bfq_log(bfqd, fmt, args...) do {} while (0) + +#else /* CONFIG_BLK_DEV_IO_TRACE */ + +#include + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq); +static struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) do { \ + blk_add_trace_msg((bfqd)->queue, "bfq%d%c %s " fmt, \ + (bfqq)->pid, \ + bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + bfqq_group(bfqq)->blkg_path, ##args); \ +} while (0) + +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do { \ + blk_add_trace_msg((bfqd)->queue, "%s " fmt, bfqg->blkg_path, ##args);\ +} while (0) + +#else /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) \ + blk_add_trace_msg((bfqd)->queue, "bfq%d%c " fmt, (bfqq)->pid, \ + bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + ##args) +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do {} while (0) + +#endif /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log(bfqd, fmt, args...) \ + blk_add_trace_msg((bfqd)->queue, "bfq " fmt, ##args) + +#endif /* CONFIG_BLK_DEV_IO_TRACE */ +#endif /* CONFIG_BFQ_REDIRECT_TO_CONSOLE */ + +/* Expiration reasons. */ +enum bfqq_expiration { + BFQ_BFQQ_TOO_IDLE = 0, /* + * queue has been idling for + * too long + */ + BFQ_BFQQ_BUDGET_TIMEOUT, /* budget took too long to be used */ + BFQ_BFQQ_BUDGET_EXHAUSTED, /* budget consumed */ + BFQ_BFQQ_NO_MORE_REQUESTS, /* the queue has no more requests */ + BFQ_BFQQ_PREEMPTED /* preemption in progress */ +}; + + +struct bfqg_stats { +#ifdef BFQ_GROUP_IOSCHED_ENABLED + /* number of ios merged */ + struct blkg_rwstat merged; + /* total time spent on device in ns, may not be accurate w/ queueing */ + struct blkg_rwstat service_time; + /* total time spent waiting in scheduler queue in ns */ + struct blkg_rwstat wait_time; + /* number of IOs queued up */ + struct blkg_rwstat queued; + /* total disk time and nr sectors dispatched by this group */ + struct blkg_stat time; + /* sum of number of ios queued across all samples */ + struct blkg_stat avg_queue_size_sum; + /* count of samples taken for average */ + struct blkg_stat avg_queue_size_samples; + /* how many times this group has been removed from service tree */ + struct blkg_stat dequeue; + /* total time spent waiting for it to be assigned a timeslice. */ + struct blkg_stat group_wait_time; + /* time spent idling for this blkcg_gq */ + struct blkg_stat idle_time; + /* total time with empty current active q with other requests queued */ + struct blkg_stat empty_time; + /* fields after this shouldn't be cleared on stat reset */ + uint64_t start_group_wait_time; + uint64_t start_idle_time; + uint64_t start_empty_time; + uint16_t flags; +#endif +}; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +/* + * struct bfq_group_data - per-blkcg storage for the blkio subsystem. + * + * @ps: @blkcg_policy_storage that this structure inherits + * @weight: weight of the bfq_group + */ +struct bfq_group_data { + /* must be the first member */ + struct blkcg_policy_data pd; + + unsigned int weight; +}; + +/** + * struct bfq_group - per (device, cgroup) data structure. + * @entity: schedulable entity to insert into the parent group sched_data. + * @sched_data: own sched_data, to contain child entities (they may be + * both bfq_queues and bfq_groups). + * @bfqd: the bfq_data for the device this group acts upon. + * @async_bfqq: array of async queues for all the tasks belonging to + * the group, one queue per ioprio value per ioprio_class, + * except for the idle class that has only one queue. + * @async_idle_bfqq: async queue for the idle class (ioprio is ignored). + * @my_entity: pointer to @entity, %NULL for the toplevel group; used + * to avoid too many special cases during group creation/ + * migration. + * @active_entities: number of active entities belonging to the group; + * unused for the root group. Used to know whether there + * are groups with more than one active @bfq_entity + * (see the comments to the function + * bfq_bfqq_may_idle()). + * @rq_pos_tree: rbtree sorted by next_request position, used when + * determining if two or more queues have interleaving + * requests (see bfq_find_close_cooperator()). + * + * Each (device, cgroup) pair has its own bfq_group, i.e., for each cgroup + * there is a set of bfq_groups, each one collecting the lower-level + * entities belonging to the group that are acting on the same device. + * + * Locking works as follows: + * o @bfqd is protected by the queue lock, RCU is used to access it + * from the readers. + * o All the other fields are protected by the @bfqd queue lock. + */ +struct bfq_group { + /* must be the first member */ + struct blkg_policy_data pd; + + /* cached path for this blkg (see comments in bfq_bic_update_cgroup) */ + char blkg_path[128]; + + /* reference counter (see comments in bfq_bic_update_cgroup) */ + int ref; + + struct bfq_entity entity; + struct bfq_sched_data sched_data; + + void *bfqd; + + struct bfq_queue *async_bfqq[2][IOPRIO_BE_NR]; + struct bfq_queue *async_idle_bfqq; + + struct bfq_entity *my_entity; + + int active_entities; + + struct rb_root rq_pos_tree; + + struct bfqg_stats stats; +}; + +#else +struct bfq_group { + struct bfq_sched_data sched_data; + + struct bfq_queue *async_bfqq[2][IOPRIO_BE_NR]; + struct bfq_queue *async_idle_bfqq; + + struct rb_root rq_pos_tree; +}; +#endif + +static struct bfq_queue *bfq_entity_to_bfqq(struct bfq_entity *entity); + +static unsigned int bfq_class_idx(struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + return bfqq ? bfqq->ioprio_class - 1 : + BFQ_DEFAULT_GRP_CLASS - 1; +} + +static struct bfq_service_tree * +bfq_entity_service_tree(struct bfq_entity *entity) +{ + struct bfq_sched_data *sched_data = entity->sched_data; + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + unsigned int idx = bfq_class_idx(entity); + + BUG_ON(idx >= BFQ_IOPRIO_CLASSES); + BUG_ON(sched_data == NULL); + + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "entity_service_tree %p %d", + sched_data->service_tree + idx, idx); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "entity_service_tree %p %d", + sched_data->service_tree + idx, idx); + } +#endif + return sched_data->service_tree + idx; +} + +static struct bfq_queue *bic_to_bfqq(struct bfq_io_cq *bic, bool is_sync) +{ + return bic->bfqq[is_sync]; +} + +static void bic_set_bfqq(struct bfq_io_cq *bic, struct bfq_queue *bfqq, + bool is_sync) +{ + bic->bfqq[is_sync] = bfqq; +} + +static struct bfq_data *bic_to_bfqd(struct bfq_io_cq *bic) +{ + return bic->icq.q->elevator->elevator_data; +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + +static struct bfq_group *bfq_bfqq_to_bfqg(struct bfq_queue *bfqq) +{ + struct bfq_entity *group_entity = bfqq->entity.parent; + + if (!group_entity) + group_entity = &bfqq->bfqd->root_group->entity; + + return container_of(group_entity, struct bfq_group, entity); +} + +#else + +static struct bfq_group *bfq_bfqq_to_bfqg(struct bfq_queue *bfqq) +{ + return bfqq->bfqd->root_group; +} + +#endif + +static void bfq_check_ioprio_change(struct bfq_io_cq *bic, struct bio *bio); +static void bfq_put_queue(struct bfq_queue *bfqq); +static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, + struct bio *bio, bool is_sync, + struct bfq_io_cq *bic); +static void bfq_end_wr_async_queues(struct bfq_data *bfqd, + struct bfq_group *bfqg); +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static void bfq_put_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg); +#endif +static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq); + +#endif /* _BFQ_H */ diff --git b/block/bfq-sched.c b/block/bfq-sched.c new file mode 100644 index 0000000..e4a2553 --- /dev/null +++ b/block/bfq-sched.c @@ -0,0 +1,2057 @@ +/* + * BFQ: Hierarchical B-WF2Q+ scheduler. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2015 Paolo Valente + * + * Copyright (C) 2016 Paolo Valente + */ + +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq); + +/** + * bfq_gt - compare two timestamps. + * @a: first ts. + * @b: second ts. + * + * Return @a > @b, dealing with wrapping correctly. + */ +static int bfq_gt(u64 a, u64 b) +{ + return (s64)(a - b) > 0; +} + +static struct bfq_entity *bfq_root_active_entity(struct rb_root *tree) +{ + struct rb_node *node = tree->rb_node; + + return rb_entry(node, struct bfq_entity, rb_node); +} + +static struct bfq_entity *bfq_lookup_next_entity(struct bfq_sched_data *sd, + bool expiration); + +static bool bfq_update_parent_budget(struct bfq_entity *next_in_service); + +/** + * bfq_update_next_in_service - update sd->next_in_service + * @sd: sched_data for which to perform the update. + * @new_entity: if not NULL, pointer to the entity whose activation, + * requeueing or repositionig triggered the invocation of + * this function. + * @expiration: id true, this function is being invoked after the + * expiration of the in-service entity + * + * This function is called to update sd->next_in_service, which, in + * its turn, may change as a consequence of the insertion or + * extraction of an entity into/from one of the active trees of + * sd. These insertions/extractions occur as a consequence of + * activations/deactivations of entities, with some activations being + * 'true' activations, and other activations being requeueings (i.e., + * implementing the second, requeueing phase of the mechanism used to + * reposition an entity in its active tree; see comments on + * __bfq_activate_entity and __bfq_requeue_entity for details). In + * both the last two activation sub-cases, new_entity points to the + * just activated or requeued entity. + * + * Returns true if sd->next_in_service changes in such a way that + * entity->parent may become the next_in_service for its parent + * entity. + */ +static bool bfq_update_next_in_service(struct bfq_sched_data *sd, + struct bfq_entity *new_entity, + bool expiration) +{ + struct bfq_entity *next_in_service = sd->next_in_service; + struct bfq_queue *bfqq; + bool parent_sched_may_change = false; + bool change_without_lookup = false; + + /* + * If this update is triggered by the activation, requeueing + * or repositiong of an entity that does not coincide with + * sd->next_in_service, then a full lookup in the active tree + * can be avoided. In fact, it is enough to check whether the + * just-modified entity has the same priority as + * sd->next_in_service, is eligible and has a lower virtual + * finish time than sd->next_in_service. If this compound + * condition holds, then the new entity becomes the new + * next_in_service. Otherwise no change is needed. + */ + if (new_entity && new_entity != sd->next_in_service) { + /* + * Flag used to decide whether to replace + * sd->next_in_service with new_entity. Tentatively + * set to true, and left as true if + * sd->next_in_service is NULL. + */ + change_without_lookup = true; + + /* + * If there is already a next_in_service candidate + * entity, then compare timestamps to decide whether + * to replace sd->service_tree with new_entity. + */ + if (next_in_service) { + unsigned int new_entity_class_idx = + bfq_class_idx(new_entity); + struct bfq_service_tree *st = + sd->service_tree + new_entity_class_idx; + + change_without_lookup = + (new_entity_class_idx == + bfq_class_idx(next_in_service) + && + !bfq_gt(new_entity->start, st->vtime) + && + bfq_gt(next_in_service->finish, + new_entity->finish)); + } + + if (change_without_lookup) { + next_in_service = new_entity; + bfqq = bfq_entity_to_bfqq(next_in_service); + + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "update_next_in_service: chose without lookup"); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(next_in_service, + struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data*)bfqg->bfqd, bfqg, + "update_next_in_service: chose without lookup"); + } +#endif + } + } + + if (!change_without_lookup) /* lookup needed */ + next_in_service = bfq_lookup_next_entity(sd, expiration); + + if (next_in_service) + parent_sched_may_change = !sd->next_in_service || + bfq_update_parent_budget(next_in_service); + + sd->next_in_service = next_in_service; + + if (!next_in_service) + return parent_sched_may_change; + + bfqq = bfq_entity_to_bfqq(next_in_service); + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "update_next_in_service: chosen this queue"); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(next_in_service, + struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "update_next_in_service: chosen this entity"); + } +#endif + return parent_sched_may_change; +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +/* both next loops stop at one of the child entities of the root group */ +#define for_each_entity(entity) \ + for (; entity ; entity = entity->parent) + +/* + * For each iteration, compute parent in advance, so as to be safe if + * entity is deallocated during the iteration. Such a deallocation may + * happen as a consequence of a bfq_put_queue that frees the bfq_queue + * containing entity. + */ +#define for_each_entity_safe(entity, parent) \ + for (; entity && ({ parent = entity->parent; 1; }); entity = parent) + +/* + * Returns true if this budget changes may let next_in_service->parent + * become the next_in_service entity for its parent entity. + */ +static bool bfq_update_parent_budget(struct bfq_entity *next_in_service) +{ + struct bfq_entity *bfqg_entity; + struct bfq_group *bfqg; + struct bfq_sched_data *group_sd; + bool ret = false; + + BUG_ON(!next_in_service); + + group_sd = next_in_service->sched_data; + + bfqg = container_of(group_sd, struct bfq_group, sched_data); + /* + * bfq_group's my_entity field is not NULL only if the group + * is not the root group. We must not touch the root entity + * as it must never become an in-service entity. + */ + bfqg_entity = bfqg->my_entity; + if (bfqg_entity) { + if (bfqg_entity->budget > next_in_service->budget) + ret = true; + bfqg_entity->budget = next_in_service->budget; + } + + return ret; +} + +/* + * This function tells whether entity stops being a candidate for next + * service, according to the restrictive definition of the field + * next_in_service. In particular, this function is invoked for an + * entity that is about to be set in service. + * + * If entity is a queue, then the entity is no longer a candidate for + * next service according to the that definition, because entity is + * about to become the in-service queue. This function then returns + * true if entity is a queue. + * + * In contrast, entity could still be a candidate for next service if + * it is not a queue, and has more than one active child. In fact, + * even if one of its children is about to be set in service, other + * active children may still be the next to serve, for the parent + * entity, even according to the above definition. As a consequence, a + * non-queue entity is not a candidate for next-service only if it has + * only one active child. And only if this condition holds, then this + * function returns true for a non-queue entity. + */ +static bool bfq_no_longer_next_in_service(struct bfq_entity *entity) +{ + struct bfq_group *bfqg; + + if (bfq_entity_to_bfqq(entity)) + return true; + + bfqg = container_of(entity, struct bfq_group, entity); + + BUG_ON(bfqg == ((struct bfq_data *)(bfqg->bfqd))->root_group); + BUG_ON(bfqg->active_entities == 0); + /* + * The field active_entities does not always contain the + * actual number of active children entities: it happens to + * not account for the in-service entity in case the latter is + * removed from its active tree (which may get done after + * invoking the function bfq_no_longer_next_in_service in + * bfq_get_next_queue). Fortunately, here, i.e., while + * bfq_no_longer_next_in_service is not yet completed in + * bfq_get_next_queue, bfq_active_extract has not yet been + * invoked, and thus active_entities still coincides with the + * actual number of active entities. + */ + if (bfqg->active_entities == 1) + return true; + + return false; +} + +#else /* BFQ_GROUP_IOSCHED_ENABLED */ +#define for_each_entity(entity) \ + for (; entity ; entity = NULL) + +#define for_each_entity_safe(entity, parent) \ + for (parent = NULL; entity ; entity = parent) + +static bool bfq_update_parent_budget(struct bfq_entity *next_in_service) +{ + return false; +} + +static bool bfq_no_longer_next_in_service(struct bfq_entity *entity) +{ + return true; +} + +#endif /* BFQ_GROUP_IOSCHED_ENABLED */ + +/* + * Shift for timestamp calculations. This actually limits the maximum + * service allowed in one timestamp delta (small shift values increase it), + * the maximum total weight that can be used for the queues in the system + * (big shift values increase it), and the period of virtual time + * wraparounds. + */ +#define WFQ_SERVICE_SHIFT 22 + +static struct bfq_queue *bfq_entity_to_bfqq(struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = NULL; + + BUG_ON(!entity); + + if (!entity->my_sched_data) + bfqq = container_of(entity, struct bfq_queue, entity); + + return bfqq; +} + + +/** + * bfq_delta - map service into the virtual time domain. + * @service: amount of service. + * @weight: scale factor (weight of an entity or weight sum). + */ +static u64 bfq_delta(unsigned long service, unsigned long weight) +{ + u64 d = (u64)service << WFQ_SERVICE_SHIFT; + + do_div(d, weight); + return d; +} + +/** + * bfq_calc_finish - assign the finish time to an entity. + * @entity: the entity to act upon. + * @service: the service to be charged to the entity. + */ +static void bfq_calc_finish(struct bfq_entity *entity, unsigned long service) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + unsigned long long start, finish, delta; + + BUG_ON(entity->weight == 0); + + entity->finish = entity->start + + bfq_delta(service, entity->weight); + + start = ((entity->start>>10)*1000)>>12; + finish = ((entity->finish>>10)*1000)>>12; + delta = ((bfq_delta(service, entity->weight)>>10)*1000)>>12; + + if (bfqq) { + bfq_log_bfqq(bfqq->bfqd, bfqq, + "calc_finish: serv %lu, w %d", + service, entity->weight); + bfq_log_bfqq(bfqq->bfqd, bfqq, + "calc_finish: start %llu, finish %llu, delta %llu", + start, finish, delta); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + } else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "calc_finish group: serv %lu, w %d", + service, entity->weight); + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "calc_finish group: start %llu, finish %llu, delta %llu", + start, finish, delta); +#endif + } +} + +/** + * bfq_entity_of - get an entity from a node. + * @node: the node field of the entity. + * + * Convert a node pointer to the relative entity. This is used only + * to simplify the logic of some functions and not as the generic + * conversion mechanism because, e.g., in the tree walking functions, + * the check for a %NULL value would be redundant. + */ +static struct bfq_entity *bfq_entity_of(struct rb_node *node) +{ + struct bfq_entity *entity = NULL; + + if (node) + entity = rb_entry(node, struct bfq_entity, rb_node); + + return entity; +} + +/** + * bfq_extract - remove an entity from a tree. + * @root: the tree root. + * @entity: the entity to remove. + */ +static void bfq_extract(struct rb_root *root, struct bfq_entity *entity) +{ + BUG_ON(entity->tree != root); + + entity->tree = NULL; + rb_erase(&entity->rb_node, root); +} + +/** + * bfq_idle_extract - extract an entity from the idle tree. + * @st: the service tree of the owning @entity. + * @entity: the entity being removed. + */ +static void bfq_idle_extract(struct bfq_service_tree *st, + struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + struct rb_node *next; + + BUG_ON(entity->tree != &st->idle); + + if (entity == st->first_idle) { + next = rb_next(&entity->rb_node); + st->first_idle = bfq_entity_of(next); + } + + if (entity == st->last_idle) { + next = rb_prev(&entity->rb_node); + st->last_idle = bfq_entity_of(next); + } + + bfq_extract(&st->idle, entity); + + if (bfqq) + list_del(&bfqq->bfqq_list); +} + +/** + * bfq_insert - generic tree insertion. + * @root: tree root. + * @entity: entity to insert. + * + * This is used for the idle and the active tree, since they are both + * ordered by finish time. + */ +static void bfq_insert(struct rb_root *root, struct bfq_entity *entity) +{ + struct bfq_entity *entry; + struct rb_node **node = &root->rb_node; + struct rb_node *parent = NULL; + + BUG_ON(entity->tree); + + while (*node) { + parent = *node; + entry = rb_entry(parent, struct bfq_entity, rb_node); + + if (bfq_gt(entry->finish, entity->finish)) + node = &parent->rb_left; + else + node = &parent->rb_right; + } + + rb_link_node(&entity->rb_node, parent, node); + rb_insert_color(&entity->rb_node, root); + + entity->tree = root; +} + +/** + * bfq_update_min - update the min_start field of a entity. + * @entity: the entity to update. + * @node: one of its children. + * + * This function is called when @entity may store an invalid value for + * min_start due to updates to the active tree. The function assumes + * that the subtree rooted at @node (which may be its left or its right + * child) has a valid min_start value. + */ +static void bfq_update_min(struct bfq_entity *entity, struct rb_node *node) +{ + struct bfq_entity *child; + + if (node) { + child = rb_entry(node, struct bfq_entity, rb_node); + if (bfq_gt(entity->min_start, child->min_start)) + entity->min_start = child->min_start; + } +} + +/** + * bfq_update_active_node - recalculate min_start. + * @node: the node to update. + * + * @node may have changed position or one of its children may have moved, + * this function updates its min_start value. The left and right subtrees + * are assumed to hold a correct min_start value. + */ +static void bfq_update_active_node(struct rb_node *node) +{ + struct bfq_entity *entity = rb_entry(node, struct bfq_entity, rb_node); + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + entity->min_start = entity->start; + bfq_update_min(entity, node->rb_right); + bfq_update_min(entity, node->rb_left); + + if (bfqq) { + bfq_log_bfqq(bfqq->bfqd, bfqq, + "update_active_node: new min_start %llu", + ((entity->min_start>>10)*1000)>>12); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + } else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "update_active_node: new min_start %llu", + ((entity->min_start>>10)*1000)>>12); +#endif + } +} + +/** + * bfq_update_active_tree - update min_start for the whole active tree. + * @node: the starting node. + * + * @node must be the deepest modified node after an update. This function + * updates its min_start using the values held by its children, assuming + * that they did not change, and then updates all the nodes that may have + * changed in the path to the root. The only nodes that may have changed + * are the ones in the path or their siblings. + */ +static void bfq_update_active_tree(struct rb_node *node) +{ + struct rb_node *parent; + +up: + bfq_update_active_node(node); + + parent = rb_parent(node); + if (!parent) + return; + + if (node == parent->rb_left && parent->rb_right) + bfq_update_active_node(parent->rb_right); + else if (parent->rb_left) + bfq_update_active_node(parent->rb_left); + + node = parent; + goto up; +} + +static void bfq_weights_tree_add(struct bfq_data *bfqd, + struct bfq_entity *entity, + struct rb_root *root); + +static void bfq_weights_tree_remove(struct bfq_data *bfqd, + struct bfq_entity *entity, + struct rb_root *root); + + +/** + * bfq_active_insert - insert an entity in the active tree of its + * group/device. + * @st: the service tree of the entity. + * @entity: the entity being inserted. + * + * The active tree is ordered by finish time, but an extra key is kept + * per each node, containing the minimum value for the start times of + * its children (and the node itself), so it's possible to search for + * the eligible node with the lowest finish time in logarithmic time. + */ +static void bfq_active_insert(struct bfq_service_tree *st, + struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + struct rb_node *node = &entity->rb_node; +#ifdef BFQ_GROUP_IOSCHED_ENABLED + struct bfq_sched_data *sd = NULL; + struct bfq_group *bfqg = NULL; + struct bfq_data *bfqd = NULL; +#endif + + bfq_insert(&st->active, entity); + + if (node->rb_left) + node = node->rb_left; + else if (node->rb_right) + node = node->rb_right; + + bfq_update_active_tree(node); + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + sd = entity->sched_data; + bfqg = container_of(sd, struct bfq_group, sched_data); + BUG_ON(!bfqg); + bfqd = (struct bfq_data *)bfqg->bfqd; +#endif + if (bfqq) + list_add(&bfqq->bfqq_list, &bfqq->bfqd->active_list); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { /* bfq_group */ + BUG_ON(!bfqd); + bfq_weights_tree_add(bfqd, entity, &bfqd->group_weights_tree); + } + if (bfqg != bfqd->root_group) { + BUG_ON(!bfqg); + BUG_ON(!bfqd); + bfqg->active_entities++; + } +#endif +} + +/** + * bfq_ioprio_to_weight - calc a weight from an ioprio. + * @ioprio: the ioprio value to convert. + */ +static unsigned short bfq_ioprio_to_weight(int ioprio) +{ + BUG_ON(ioprio < 0 || ioprio >= IOPRIO_BE_NR); + return (IOPRIO_BE_NR - ioprio) * BFQ_WEIGHT_CONVERSION_COEFF; +} + +/** + * bfq_weight_to_ioprio - calc an ioprio from a weight. + * @weight: the weight value to convert. + * + * To preserve as much as possible the old only-ioprio user interface, + * 0 is used as an escape ioprio value for weights (numerically) equal or + * larger than IOPRIO_BE_NR * BFQ_WEIGHT_CONVERSION_COEFF. + */ +static unsigned short bfq_weight_to_ioprio(int weight) +{ + BUG_ON(weight < BFQ_MIN_WEIGHT || weight > BFQ_MAX_WEIGHT); + return IOPRIO_BE_NR * BFQ_WEIGHT_CONVERSION_COEFF - weight < 0 ? + 0 : IOPRIO_BE_NR * BFQ_WEIGHT_CONVERSION_COEFF - weight; +} + +static void bfq_get_entity(struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + if (bfqq) { + bfqq->ref++; + bfq_log_bfqq(bfqq->bfqd, bfqq, "get_entity: %p %d", + bfqq, bfqq->ref); + } +} + +/** + * bfq_find_deepest - find the deepest node that an extraction can modify. + * @node: the node being removed. + * + * Do the first step of an extraction in an rb tree, looking for the + * node that will replace @node, and returning the deepest node that + * the following modifications to the tree can touch. If @node is the + * last node in the tree return %NULL. + */ +static struct rb_node *bfq_find_deepest(struct rb_node *node) +{ + struct rb_node *deepest; + + if (!node->rb_right && !node->rb_left) + deepest = rb_parent(node); + else if (!node->rb_right) + deepest = node->rb_left; + else if (!node->rb_left) + deepest = node->rb_right; + else { + deepest = rb_next(node); + if (deepest->rb_right) + deepest = deepest->rb_right; + else if (rb_parent(deepest) != node) + deepest = rb_parent(deepest); + } + + return deepest; +} + +/** + * bfq_active_extract - remove an entity from the active tree. + * @st: the service_tree containing the tree. + * @entity: the entity being removed. + */ +static void bfq_active_extract(struct bfq_service_tree *st, + struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + struct rb_node *node; +#ifdef BFQ_GROUP_IOSCHED_ENABLED + struct bfq_sched_data *sd = NULL; + struct bfq_group *bfqg = NULL; + struct bfq_data *bfqd = NULL; +#endif + + node = bfq_find_deepest(&entity->rb_node); + bfq_extract(&st->active, entity); + + if (node) + bfq_update_active_tree(node); + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + sd = entity->sched_data; + bfqg = container_of(sd, struct bfq_group, sched_data); + BUG_ON(!bfqg); + bfqd = (struct bfq_data *)bfqg->bfqd; +#endif + if (bfqq) + list_del(&bfqq->bfqq_list); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { /* bfq_group */ + BUG_ON(!bfqd); + bfq_weights_tree_remove(bfqd, entity, + &bfqd->group_weights_tree); + } + if (bfqg != bfqd->root_group) { + BUG_ON(!bfqg); + BUG_ON(!bfqd); + BUG_ON(!bfqg->active_entities); + bfqg->active_entities--; + } +#endif +} + +/** + * bfq_idle_insert - insert an entity into the idle tree. + * @st: the service tree containing the tree. + * @entity: the entity to insert. + */ +static void bfq_idle_insert(struct bfq_service_tree *st, + struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + struct bfq_entity *first_idle = st->first_idle; + struct bfq_entity *last_idle = st->last_idle; + + if (!first_idle || bfq_gt(first_idle->finish, entity->finish)) + st->first_idle = entity; + if (!last_idle || bfq_gt(entity->finish, last_idle->finish)) + st->last_idle = entity; + + bfq_insert(&st->idle, entity); + + if (bfqq) + list_add(&bfqq->bfqq_list, &bfqq->bfqd->idle_list); +} + +/** + * bfq_forget_entity - do not consider entity any longer for scheduling + * @st: the service tree. + * @entity: the entity being removed. + * @is_in_service: true if entity is currently the in-service entity. + * + * Forget everything about @entity. In addition, if entity represents + * a queue, and the latter is not in service, then release the service + * reference to the queue (the one taken through bfq_get_entity). In + * fact, in this case, there is really no more service reference to + * the queue, as the latter is also outside any service tree. If, + * instead, the queue is in service, then __bfq_bfqd_reset_in_service + * will take care of putting the reference when the queue finally + * stops being served. + */ +static void bfq_forget_entity(struct bfq_service_tree *st, + struct bfq_entity *entity, + bool is_in_service) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + BUG_ON(!entity->on_st); + + entity->on_st = false; + st->wsum -= entity->weight; + if (bfqq && !is_in_service) { + bfq_log_bfqq(bfqq->bfqd, bfqq, "forget_entity (before): %p %d", + bfqq, bfqq->ref); + bfq_put_queue(bfqq); + } +} + +/** + * bfq_put_idle_entity - release the idle tree ref of an entity. + * @st: service tree for the entity. + * @entity: the entity being released. + */ +static void bfq_put_idle_entity(struct bfq_service_tree *st, + struct bfq_entity *entity) +{ + bfq_idle_extract(st, entity); + bfq_forget_entity(st, entity, + entity == entity->sched_data->in_service_entity); +} + +/** + * bfq_forget_idle - update the idle tree if necessary. + * @st: the service tree to act upon. + * + * To preserve the global O(log N) complexity we only remove one entry here; + * as the idle tree will not grow indefinitely this can be done safely. + */ +static void bfq_forget_idle(struct bfq_service_tree *st) +{ + struct bfq_entity *first_idle = st->first_idle; + struct bfq_entity *last_idle = st->last_idle; + + if (RB_EMPTY_ROOT(&st->active) && last_idle && + !bfq_gt(last_idle->finish, st->vtime)) { + /* + * Forget the whole idle tree, increasing the vtime past + * the last finish time of idle entities. + */ + st->vtime = last_idle->finish; + } + + if (first_idle && !bfq_gt(first_idle->finish, st->vtime)) + bfq_put_idle_entity(st, first_idle); +} + +/* + * Update weight and priority of entity. If update_class_too is true, + * then update the ioprio_class of entity too. + * + * The reason why the update of ioprio_class is controlled through the + * last parameter is as follows. Changing the ioprio class of an + * entity implies changing the destination service trees for that + * entity. If such a change occurred when the entity is already on one + * of the service trees for its previous class, then the state of the + * entity would become more complex: none of the new possible service + * trees for the entity, according to bfq_entity_service_tree(), would + * match any of the possible service trees on which the entity + * is. Complex operations involving these trees, such as entity + * activations and deactivations, should take into account this + * additional complexity. To avoid this issue, this function is + * invoked with update_class_too unset in the points in the code where + * entity may happen to be on some tree. + */ +static struct bfq_service_tree * +__bfq_entity_update_weight_prio(struct bfq_service_tree *old_st, + struct bfq_entity *entity, + bool update_class_too) +{ + struct bfq_service_tree *new_st = old_st; + + if (entity->prio_changed) { + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + unsigned int prev_weight, new_weight; + struct bfq_data *bfqd = NULL; + struct rb_root *root; +#ifdef BFQ_GROUP_IOSCHED_ENABLED + struct bfq_sched_data *sd; + struct bfq_group *bfqg; +#endif + + if (bfqq) + bfqd = bfqq->bfqd; +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + sd = entity->my_sched_data; + bfqg = container_of(sd, struct bfq_group, sched_data); + BUG_ON(!bfqg); + bfqd = (struct bfq_data *)bfqg->bfqd; + BUG_ON(!bfqd); + } +#endif + + BUG_ON(entity->tree && update_class_too); + BUG_ON(old_st->wsum < entity->weight); + old_st->wsum -= entity->weight; + + if (entity->new_weight != entity->orig_weight) { + if (entity->new_weight < BFQ_MIN_WEIGHT || + entity->new_weight > BFQ_MAX_WEIGHT) { + pr_crit("update_weight_prio: new_weight %d\n", + entity->new_weight); + if (entity->new_weight < BFQ_MIN_WEIGHT) + entity->new_weight = BFQ_MIN_WEIGHT; + else + entity->new_weight = BFQ_MAX_WEIGHT; + } + entity->orig_weight = entity->new_weight; + if (bfqq) + bfqq->ioprio = + bfq_weight_to_ioprio(entity->orig_weight); + } + + if (bfqq && update_class_too) + bfqq->ioprio_class = bfqq->new_ioprio_class; + + /* + * Reset prio_changed only if the ioprio_class change + * is not pending any longer. + */ + if (!bfqq || bfqq->ioprio_class == bfqq->new_ioprio_class) + entity->prio_changed = 0; + + /* + * NOTE: here we may be changing the weight too early, + * this will cause unfairness. The correct approach + * would have required additional complexity to defer + * weight changes to the proper time instants (i.e., + * when entity->finish <= old_st->vtime). + */ + new_st = bfq_entity_service_tree(entity); + + prev_weight = entity->weight; + new_weight = entity->orig_weight * + (bfqq ? bfqq->wr_coeff : 1); + /* + * If the weight of the entity changes, remove the entity + * from its old weight counter (if there is a counter + * associated with the entity), and add it to the counter + * associated with its new weight. + */ + if (prev_weight != new_weight) { + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "weight changed %d %d(%d %d)", + prev_weight, new_weight, + entity->orig_weight, + bfqq->wr_coeff); + + root = bfqq ? &bfqd->queue_weights_tree : + &bfqd->group_weights_tree; + bfq_weights_tree_remove(bfqd, entity, root); + } + entity->weight = new_weight; + /* + * Add the entity to its weights tree only if it is + * not associated with a weight-raised queue. + */ + if (prev_weight != new_weight && + (bfqq ? bfqq->wr_coeff == 1 : 1)) + /* If we get here, root has been initialized. */ + bfq_weights_tree_add(bfqd, entity, root); + + new_st->wsum += entity->weight; + + if (new_st != old_st) { + BUG_ON(!update_class_too); + entity->start = new_st->vtime; + } + } + + return new_st; +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static void bfqg_stats_set_start_empty_time(struct bfq_group *bfqg); +#endif + +/** + * bfq_bfqq_served - update the scheduler status after selection for + * service. + * @bfqq: the queue being served. + * @served: bytes to transfer. + * + * NOTE: this can be optimized, as the timestamps of upper level entities + * are synchronized every time a new bfqq is selected for service. By now, + * we keep it to better check consistency. + */ +static void bfq_bfqq_served(struct bfq_queue *bfqq, int served) +{ + struct bfq_entity *entity = &bfqq->entity; + struct bfq_service_tree *st; + + for_each_entity(entity) { + st = bfq_entity_service_tree(entity); + + entity->service += served; + + BUG_ON(st->wsum == 0); + + st->vtime += bfq_delta(served, st->wsum); + bfq_forget_idle(st); + } +#ifdef BFQ_GROUP_IOSCHED_ENABLED + bfqg_stats_set_start_empty_time(bfqq_group(bfqq)); +#endif + st = bfq_entity_service_tree(&bfqq->entity); + bfq_log_bfqq(bfqq->bfqd, bfqq, "bfqq_served %d secs, vtime %llu on %p", + served, ((st->vtime>>10)*1000)>>12, st); +} + +/** + * bfq_bfqq_charge_time - charge an amount of service equivalent to the length + * of the time interval during which bfqq has been in + * service. + * @bfqd: the device + * @bfqq: the queue that needs a service update. + * @time_ms: the amount of time during which the queue has received service + * + * If a queue does not consume its budget fast enough, then providing + * the queue with service fairness may impair throughput, more or less + * severely. For this reason, queues that consume their budget slowly + * are provided with time fairness instead of service fairness. This + * goal is achieved through the BFQ scheduling engine, even if such an + * engine works in the service, and not in the time domain. The trick + * is charging these queues with an inflated amount of service, equal + * to the amount of service that they would have received during their + * service slot if they had been fast, i.e., if their requests had + * been dispatched at a rate equal to the estimated peak rate. + * + * It is worth noting that time fairness can cause important + * distortions in terms of bandwidth distribution, on devices with + * internal queueing. The reason is that I/O requests dispatched + * during the service slot of a queue may be served after that service + * slot is finished, and may have a total processing time loosely + * correlated with the duration of the service slot. This is + * especially true for short service slots. + */ +static void bfq_bfqq_charge_time(struct bfq_data *bfqd, struct bfq_queue *bfqq, + unsigned long time_ms) +{ + struct bfq_entity *entity = &bfqq->entity; + int tot_serv_to_charge = entity->service; + unsigned int timeout_ms = jiffies_to_msecs(bfq_timeout); + + if (time_ms > 0 && time_ms < timeout_ms) + tot_serv_to_charge = + (bfqd->bfq_max_budget * time_ms) / timeout_ms; + + if (tot_serv_to_charge < entity->service) + tot_serv_to_charge = entity->service; + + bfq_log_bfqq(bfqq->bfqd, bfqq, + "charge_time: %lu/%u ms, %d/%d/%d sectors", + time_ms, timeout_ms, entity->service, + tot_serv_to_charge, entity->budget); + + /* Increase budget to avoid inconsistencies */ + if (tot_serv_to_charge > entity->budget) + entity->budget = tot_serv_to_charge; + + bfq_bfqq_served(bfqq, + max_t(int, 0, tot_serv_to_charge - entity->service)); +} + +static void bfq_update_fin_time_enqueue(struct bfq_entity *entity, + struct bfq_service_tree *st, + bool backshifted) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + struct bfq_sched_data *sd = entity->sched_data; + + /* + * When this function is invoked, entity is not in any service + * tree, then it is safe to invoke next function with the last + * parameter set (see the comments on the function). + */ + BUG_ON(entity->tree); + st = __bfq_entity_update_weight_prio(st, entity, true); + bfq_calc_finish(entity, entity->budget); + + /* + * If some queues enjoy backshifting for a while, then their + * (virtual) finish timestamps may happen to become lower and + * lower than the system virtual time. In particular, if + * these queues often happen to be idle for short time + * periods, and during such time periods other queues with + * higher timestamps happen to be busy, then the backshifted + * timestamps of the former queues can become much lower than + * the system virtual time. In fact, to serve the queues with + * higher timestamps while the ones with lower timestamps are + * idle, the system virtual time may be pushed-up to much + * higher values than the finish timestamps of the idle + * queues. As a consequence, the finish timestamps of all new + * or newly activated queues may end up being much larger than + * those of lucky queues with backshifted timestamps. The + * latter queues may then monopolize the device for a lot of + * time. This would simply break service guarantees. + * + * To reduce this problem, push up a little bit the + * backshifted timestamps of the queue associated with this + * entity (only a queue can happen to have the backshifted + * flag set): just enough to let the finish timestamp of the + * queue be equal to the current value of the system virtual + * time. This may introduce a little unfairness among queues + * with backshifted timestamps, but it does not break + * worst-case fairness guarantees. + * + * As a special case, if bfqq is weight-raised, push up + * timestamps much less, to keep very low the probability that + * this push up causes the backshifted finish timestamps of + * weight-raised queues to become higher than the backshifted + * finish timestamps of non weight-raised queues. + */ + if (backshifted && bfq_gt(st->vtime, entity->finish)) { + unsigned long delta = st->vtime - entity->finish; + + if (bfqq) + delta /= bfqq->wr_coeff; + + entity->start += delta; + entity->finish += delta; + + if (bfqq) { + bfq_log_bfqq(bfqq->bfqd, bfqq, + "update_fin_time_enqueue: new queue finish %llu", + ((entity->finish>>10)*1000)>>12); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + } else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "update_fin_time_enqueue: new group finish %llu", + ((entity->finish>>10)*1000)>>12); +#endif + } + } + + bfq_active_insert(st, entity); + + if (bfqq) { + bfq_log_bfqq(bfqq->bfqd, bfqq, + "update_fin_time_enqueue: queue %seligible in st %p", + entity->start <= st->vtime ? "" : "non ", st); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + } else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "update_fin_time_enqueue: group %seligible in st %p", + entity->start <= st->vtime ? "" : "non ", st); +#endif + } + BUG_ON(RB_EMPTY_ROOT(&st->active)); + BUG_ON(&st->active != &sd->service_tree->active && + &st->active != &(sd->service_tree+1)->active && + &st->active != &(sd->service_tree+2)->active); +} + +/** + * __bfq_activate_entity - handle activation of entity. + * @entity: the entity being activated. + * @non_blocking_wait_rq: true if entity was waiting for a request + * + * Called for a 'true' activation, i.e., if entity is not active and + * one of its children receives a new request. + * + * Basically, this function updates the timestamps of entity and + * inserts entity into its active tree, ater possibly extracting it + * from its idle tree. + */ +static void __bfq_activate_entity(struct bfq_entity *entity, + bool non_blocking_wait_rq) +{ + struct bfq_sched_data *sd = entity->sched_data; + struct bfq_service_tree *st = bfq_entity_service_tree(entity); + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + bool backshifted = false; + unsigned long long min_vstart; + + BUG_ON(!sd); + BUG_ON(!st); + + /* See comments on bfq_fqq_update_budg_for_activation */ + if (non_blocking_wait_rq && bfq_gt(st->vtime, entity->finish)) { + backshifted = true; + min_vstart = entity->finish; + } else + min_vstart = st->vtime; + + if (entity->tree == &st->idle) { + /* + * Must be on the idle tree, bfq_idle_extract() will + * check for that. + */ + bfq_idle_extract(st, entity); + BUG_ON(entity->tree); + entity->start = bfq_gt(min_vstart, entity->finish) ? + min_vstart : entity->finish; + } else { + BUG_ON(entity->tree); + /* + * The finish time of the entity may be invalid, and + * it is in the past for sure, otherwise the queue + * would have been on the idle tree. + */ + entity->start = min_vstart; + st->wsum += entity->weight; + /* + * entity is about to be inserted into a service tree, + * and then set in service: get a reference to make + * sure entity does not disappear until it is no + * longer in service or scheduled for service. + */ + bfq_get_entity(entity); + + BUG_ON(entity->on_st && bfqq); + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + if (entity->on_st && !bfqq) { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, + entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, + bfqg, + "activate bug, class %d in_service %p", + bfq_class_idx(entity), sd->in_service_entity); + } +#endif + BUG_ON(entity->on_st && !bfqq); + entity->on_st = true; + } + + bfq_update_fin_time_enqueue(entity, st, backshifted); +} + +/** + * __bfq_requeue_entity - handle requeueing or repositioning of an entity. + * @entity: the entity being requeued or repositioned. + * + * Requeueing is needed if this entity stops being served, which + * happens if a leaf descendant entity has expired. On the other hand, + * repositioning is needed if the next_inservice_entity for the child + * entity has changed. See the comments inside the function for + * details. + * + * Basically, this function: 1) removes entity from its active tree if + * present there, 2) updates the timestamps of entity and 3) inserts + * entity back into its active tree (in the new, right position for + * the new values of the timestamps). + */ +static void __bfq_requeue_entity(struct bfq_entity *entity) +{ + struct bfq_sched_data *sd = entity->sched_data; + struct bfq_service_tree *st = bfq_entity_service_tree(entity); + + BUG_ON(!sd); + BUG_ON(!st); + + BUG_ON(entity != sd->in_service_entity && + entity->tree != &st->active); + + if (entity == sd->in_service_entity) { + /* + * We are requeueing the current in-service entity, + * which may have to be done for one of the following + * reasons: + * - entity represents the in-service queue, and the + * in-service queue is being requeued after an + * expiration; + * - entity represents a group, and its budget has + * changed because one of its child entities has + * just been either activated or requeued for some + * reason; the timestamps of the entity need then to + * be updated, and the entity needs to be enqueued + * or repositioned accordingly. + * + * In particular, before requeueing, the start time of + * the entity must be moved forward to account for the + * service that the entity has received while in + * service. This is done by the next instructions. The + * finish time will then be updated according to this + * new value of the start time, and to the budget of + * the entity. + */ + bfq_calc_finish(entity, entity->service); + entity->start = entity->finish; + BUG_ON(entity->tree && entity->tree == &st->idle); + BUG_ON(entity->tree && entity->tree != &st->active); + /* + * In addition, if the entity had more than one child + * when set in service, then it was not extracted from + * the active tree. This implies that the position of + * the entity in the active tree may need to be + * changed now, because we have just updated the start + * time of the entity, and we will update its finish + * time in a moment (the requeueing is then, more + * precisely, a repositioning in this case). To + * implement this repositioning, we: 1) dequeue the + * entity here, 2) update the finish time and requeue + * the entity according to the new timestamps below. + */ + if (entity->tree) + bfq_active_extract(st, entity); + } else { /* The entity is already active, and not in service */ + /* + * In this case, this function gets called only if the + * next_in_service entity below this entity has + * changed, and this change has caused the budget of + * this entity to change, which, finally implies that + * the finish time of this entity must be + * updated. Such an update may cause the scheduling, + * i.e., the position in the active tree, of this + * entity to change. We handle this change by: 1) + * dequeueing the entity here, 2) updating the finish + * time and requeueing the entity according to the new + * timestamps below. This is the same approach as the + * non-extracted-entity sub-case above. + */ + bfq_active_extract(st, entity); + } + + bfq_update_fin_time_enqueue(entity, st, false); +} + +static void __bfq_activate_requeue_entity(struct bfq_entity *entity, + struct bfq_sched_data *sd, + bool non_blocking_wait_rq) +{ + struct bfq_service_tree *st = bfq_entity_service_tree(entity); + + if (sd->in_service_entity == entity || entity->tree == &st->active) + /* + * in service or already queued on the active tree, + * requeue or reposition + */ + __bfq_requeue_entity(entity); + else + /* + * Not in service and not queued on its active tree: + * the activity is idle and this is a true activation. + */ + __bfq_activate_entity(entity, non_blocking_wait_rq); +} + + +/** + * bfq_activate_requeue_entity - activate or requeue an entity representing a bfq_queue, + * and activate, requeue or reposition all ancestors + * for which such an update becomes necessary. + * @entity: the entity to activate. + * @non_blocking_wait_rq: true if this entity was waiting for a request + * @requeue: true if this is a requeue, which implies that bfqq is + * being expired; thus ALL its ancestors stop being served and must + * therefore be requeued + * @expiration: true if this function is being invoked in the expiration path + * of the in-service queue + */ +static void bfq_activate_requeue_entity(struct bfq_entity *entity, + bool non_blocking_wait_rq, + bool requeue, bool expiration) +{ + struct bfq_sched_data *sd; + + for_each_entity(entity) { + BUG_ON(!entity); + sd = entity->sched_data; + __bfq_activate_requeue_entity(entity, sd, non_blocking_wait_rq); + + BUG_ON(RB_EMPTY_ROOT(&sd->service_tree->active) && + RB_EMPTY_ROOT(&(sd->service_tree+1)->active) && + RB_EMPTY_ROOT(&(sd->service_tree+2)->active)); + + if (!bfq_update_next_in_service(sd, entity, expiration) && + !requeue) { + BUG_ON(!sd->next_in_service); + break; + } + BUG_ON(!sd->next_in_service); + } +} + +/** + * __bfq_deactivate_entity - deactivate an entity from its service tree. + * @entity: the entity to deactivate. + * @ins_into_idle_tree: if false, the entity will not be put into the + * idle tree. + * + * Deactivates an entity, independently of its previous state. Must + * be invoked only if entity is on a service tree. Extracts the entity + * from that tree, and if necessary and allowed, puts it into the idle + * tree. + */ +static bool __bfq_deactivate_entity(struct bfq_entity *entity, + bool ins_into_idle_tree) +{ + struct bfq_sched_data *sd = entity->sched_data; + struct bfq_service_tree *st; + bool is_in_service; + + if (!entity->on_st) { /* entity never activated, or already inactive */ + BUG_ON(sd && entity == sd->in_service_entity); + return false; + } + + /* + * If we get here, then entity is active, which implies that + * bfq_group_set_parent has already been invoked for the group + * represented by entity. Therefore, the field + * entity->sched_data has been set, and we can safely use it. + */ + st = bfq_entity_service_tree(entity); + is_in_service = entity == sd->in_service_entity; + + BUG_ON(is_in_service && entity->tree && entity->tree != &st->active); + + if (is_in_service) { + bfq_calc_finish(entity, entity->service); + sd->in_service_entity = NULL; + } + + if (entity->tree == &st->active) + bfq_active_extract(st, entity); + else if (!is_in_service && entity->tree == &st->idle) + bfq_idle_extract(st, entity); + else if (entity->tree) + BUG(); + + if (!ins_into_idle_tree || !bfq_gt(entity->finish, st->vtime)) + bfq_forget_entity(st, entity, is_in_service); + else + bfq_idle_insert(st, entity); + + return true; +} + +/** + * bfq_deactivate_entity - deactivate an entity representing a bfq_queue. + * @entity: the entity to deactivate. + * @ins_into_idle_tree: true if the entity can be put into the idle tree + * @expiration: true if this function is being invoked in the expiration path + * of the in-service queue + */ +static void bfq_deactivate_entity(struct bfq_entity *entity, + bool ins_into_idle_tree, + bool expiration) +{ + struct bfq_sched_data *sd; + struct bfq_entity *parent = NULL; + + for_each_entity_safe(entity, parent) { + sd = entity->sched_data; + + BUG_ON(sd == NULL); /* + * It would mean that this is the + * root group. + */ + + BUG_ON(expiration && entity != sd->in_service_entity); + + BUG_ON(entity != sd->in_service_entity && + entity->tree == + &bfq_entity_service_tree(entity)->active && + !sd->next_in_service); + + if (!__bfq_deactivate_entity(entity, ins_into_idle_tree)) { + /* + * entity is not in any tree any more, so + * this deactivation is a no-op, and there is + * nothing to change for upper-level entities + * (in case of expiration, this can never + * happen). + */ + BUG_ON(expiration); /* + * entity cannot be already out of + * any tree + */ + return; + } + + if (sd->next_in_service == entity) + /* + * entity was the next_in_service entity, + * then, since entity has just been + * deactivated, a new one must be found. + */ + bfq_update_next_in_service(sd, NULL, expiration); + + if (sd->next_in_service || sd->in_service_entity) { + /* + * The parent entity is still active, because + * either next_in_service or in_service_entity + * is not NULL. So, no further upwards + * deactivation must be performed. Yet, + * next_in_service has changed. Then the + * schedule does need to be updated upwards. + * + * NOTE If in_service_entity is not NULL, then + * next_in_service may happen to be NULL, + * although the parent entity is evidently + * active. This happens if 1) the entity + * pointed by in_service_entity is the only + * active entity in the parent entity, and 2) + * according to the definition of + * next_in_service, the in_service_entity + * cannot be considered as + * next_in_service. See the comments on the + * definition of next_in_service for details. + */ + BUG_ON(sd->next_in_service == entity); + BUG_ON(sd->in_service_entity == entity); + break; + } + + /* + * If we get here, then the parent is no more + * backlogged and we need to propagate the + * deactivation upwards. Thus let the loop go on. + */ + + /* + * Also let parent be queued into the idle tree on + * deactivation, to preserve service guarantees, and + * assuming that who invoked this function does not + * need parent entities too to be removed completely. + */ + ins_into_idle_tree = true; + } + + /* + * If the deactivation loop is fully executed, then there are + * no more entities to touch and next loop is not executed at + * all. Otherwise, requeue remaining entities if they are + * about to stop receiving service, or reposition them if this + * is not the case. + */ + entity = parent; + for_each_entity(entity) { + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + /* + * Invoke __bfq_requeue_entity on entity, even if + * already active, to requeue/reposition it in the + * active tree (because sd->next_in_service has + * changed) + */ + __bfq_requeue_entity(entity); + + sd = entity->sched_data; + BUG_ON(expiration && sd->in_service_entity != entity); + + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "invoking udpdate_next for this queue"); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(entity, + struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "invoking udpdate_next for this entity"); + } +#endif + if (!bfq_update_next_in_service(sd, entity, expiration) && + !expiration) + /* + * next_in_service unchanged or not causing + * any change in entity->parent->sd, and no + * requeueing needed for expiration: stop + * here. + */ + break; + } +} + +/** + * bfq_calc_vtime_jump - compute the value to which the vtime should jump, + * if needed, to have at least one entity eligible. + * @st: the service tree to act upon. + * + * Assumes that st is not empty. + */ +static u64 bfq_calc_vtime_jump(struct bfq_service_tree *st) +{ + struct bfq_entity *root_entity = bfq_root_active_entity(&st->active); + + if (bfq_gt(root_entity->min_start, st->vtime)) { + struct bfq_queue *bfqq = bfq_entity_to_bfqq(root_entity); + + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "calc_vtime_jump: new value %llu", + ((root_entity->min_start>>10)*1000)>>12); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(root_entity, struct bfq_group, + entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "calc_vtime_jump: new value %llu", + ((root_entity->min_start>>10)*1000)>>12); + } +#endif + return root_entity->min_start; + } + return st->vtime; +} + +static void bfq_update_vtime(struct bfq_service_tree *st, u64 new_value) +{ + if (new_value > st->vtime) { + st->vtime = new_value; + bfq_forget_idle(st); + } +} + +/** + * bfq_first_active_entity - find the eligible entity with + * the smallest finish time + * @st: the service tree to select from. + * @vtime: the system virtual to use as a reference for eligibility + * + * This function searches the first schedulable entity, starting from the + * root of the tree and going on the left every time on this side there is + * a subtree with at least one eligible (start >= vtime) entity. The path on + * the right is followed only if a) the left subtree contains no eligible + * entities and b) no eligible entity has been found yet. + */ +static struct bfq_entity *bfq_first_active_entity(struct bfq_service_tree *st, + u64 vtime) +{ + struct bfq_entity *entry, *first = NULL; + struct rb_node *node = st->active.rb_node; + + while (node) { + entry = rb_entry(node, struct bfq_entity, rb_node); +left: + if (!bfq_gt(entry->start, vtime)) + first = entry; + + BUG_ON(bfq_gt(entry->min_start, vtime)); + + if (node->rb_left) { + entry = rb_entry(node->rb_left, + struct bfq_entity, rb_node); + if (!bfq_gt(entry->min_start, vtime)) { + node = node->rb_left; + goto left; + } + } + if (first) + break; + node = node->rb_right; + } + + BUG_ON(!first && !RB_EMPTY_ROOT(&st->active)); + return first; +} + +/** + * __bfq_lookup_next_entity - return the first eligible entity in @st. + * @st: the service tree. + * + * If there is no in-service entity for the sched_data st belongs to, + * then return the entity that will be set in service if: + * 1) the parent entity this st belongs to is set in service; + * 2) no entity belonging to such parent entity undergoes a state change + * that would influence the timestamps of the entity (e.g., becomes idle, + * becomes backlogged, changes its budget, ...). + * + * In this first case, update the virtual time in @st too (see the + * comments on this update inside the function). + * + * In constrast, if there is an in-service entity, then return the + * entity that would be set in service if not only the above + * conditions, but also the next one held true: the currently + * in-service entity, on expiration, + * 1) gets a finish time equal to the current one, or + * 2) is not eligible any more, or + * 3) is idle. + */ +static struct bfq_entity * +__bfq_lookup_next_entity(struct bfq_service_tree *st, bool in_service) +{ + struct bfq_entity *entity; + u64 new_vtime; + struct bfq_queue *bfqq; + + if (RB_EMPTY_ROOT(&st->active)) + return NULL; + + /* + * Get the value of the system virtual time for which at + * least one entity is eligible. + */ + new_vtime = bfq_calc_vtime_jump(st); + + /* + * If there is no in-service entity for the sched_data this + * active tree belongs to, then push the system virtual time + * up to the value that guarantees that at least one entity is + * eligible. If, instead, there is an in-service entity, then + * do not make any such update, because there is already an + * eligible entity, namely the in-service one (even if the + * entity is not on st, because it was extracted when set in + * service). + */ + if (!in_service) + bfq_update_vtime(st, new_vtime); + + entity = bfq_first_active_entity(st, new_vtime); + BUG_ON(bfq_gt(entity->start, new_vtime)); + + /* Log some information */ + bfqq = bfq_entity_to_bfqq(entity); + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "__lookup_next: start %llu vtime %llu st %p", + ((entity->start>>10)*1000)>>12, + ((new_vtime>>10)*1000)>>12, st); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "__lookup_next: start %llu vtime %llu (%llu) st %p", + ((entity->start>>10)*1000)>>12, + ((st->vtime>>10)*1000)>>12, + ((new_vtime>>10)*1000)>>12, st); + } +#endif + + BUG_ON(!entity); + + return entity; +} + +/** + * bfq_lookup_next_entity - return the first eligible entity in @sd. + * @sd: the sched_data. + * @expiration: true if we are on the expiration path of the in-service queue + * + * This function is invoked when there has been a change in the trees + * for sd, and we need to know what is the new next entity to serve + * after this change. + */ +static struct bfq_entity *bfq_lookup_next_entity(struct bfq_sched_data *sd, + bool expiration) +{ + struct bfq_service_tree *st = sd->service_tree; + struct bfq_service_tree *idle_class_st = st + (BFQ_IOPRIO_CLASSES - 1); + struct bfq_entity *entity = NULL; + struct bfq_queue *bfqq; + int class_idx = 0; + + BUG_ON(!sd); + BUG_ON(!st); + /* + * Choose from idle class, if needed to guarantee a minimum + * bandwidth to this class (and if there is some active entity + * in idle class). This should also mitigate + * priority-inversion problems in case a low priority task is + * holding file system resources. + */ + if (time_is_before_jiffies(sd->bfq_class_idle_last_service + + BFQ_CL_IDLE_TIMEOUT)) { + if (!RB_EMPTY_ROOT(&idle_class_st->active)) + class_idx = BFQ_IOPRIO_CLASSES - 1; + /* About to be served if backlogged, or not yet backlogged */ + sd->bfq_class_idle_last_service = jiffies; + } + + /* + * Find the next entity to serve for the highest-priority + * class, unless the idle class needs to be served. + */ + for (; class_idx < BFQ_IOPRIO_CLASSES; class_idx++) { + /* + * If expiration is true, then bfq_lookup_next_entity + * is being invoked as a part of the expiration path + * of the in-service queue. In this case, even if + * sd->in_service_entity is not NULL, + * sd->in_service_entiy at this point is actually not + * in service any more, and, if needed, has already + * been properly queued or requeued into the right + * tree. The reason why sd->in_service_entity is still + * not NULL here, even if expiration is true, is that + * sd->in_service_entiy is reset as a last step in the + * expiration path. So, if expiration is true, tell + * __bfq_lookup_next_entity that there is no + * sd->in_service_entity. + */ + entity = __bfq_lookup_next_entity(st + class_idx, + sd->in_service_entity && + !expiration); + + if (entity) + break; + } + + BUG_ON(!entity && + (!RB_EMPTY_ROOT(&st->active) || !RB_EMPTY_ROOT(&(st+1)->active) || + !RB_EMPTY_ROOT(&(st+2)->active))); + + if (!entity) + return NULL; + + /* Log some information */ + bfqq = bfq_entity_to_bfqq(entity); + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, "chosen from st %p %d", + st + class_idx, class_idx); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "chosen from st %p %d", + st + class_idx, class_idx); + } +#endif + + return entity; +} + +static bool next_queue_may_preempt(struct bfq_data *bfqd) +{ + struct bfq_sched_data *sd = &bfqd->root_group->sched_data; + + return sd->next_in_service != sd->in_service_entity; +} + +/* + * Get next queue for service. + */ +static struct bfq_queue *bfq_get_next_queue(struct bfq_data *bfqd) +{ + struct bfq_entity *entity = NULL; + struct bfq_sched_data *sd; + struct bfq_queue *bfqq; + + BUG_ON(bfqd->in_service_queue); + + if (bfqd->busy_queues == 0) + return NULL; + + /* + * Traverse the path from the root to the leaf entity to + * serve. Set in service all the entities visited along the + * way. + */ + sd = &bfqd->root_group->sched_data; + for (; sd ; sd = entity->my_sched_data) { +#ifdef BFQ_GROUP_IOSCHED_ENABLED + if (entity) { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg(bfqd, bfqg, + "get_next_queue: lookup in this group"); + if (!sd->next_in_service) + pr_crit("get_next_queue: lookup in this group"); + } else { + bfq_log_bfqg(bfqd, bfqd->root_group, + "get_next_queue: lookup in root group"); + if (!sd->next_in_service) + pr_crit("get_next_queue: lookup in root group"); + } +#endif + + BUG_ON(!sd->next_in_service); + + /* + * WARNING. We are about to set the in-service entity + * to sd->next_in_service, i.e., to the (cached) value + * returned by bfq_lookup_next_entity(sd) the last + * time it was invoked, i.e., the last time when the + * service order in sd changed as a consequence of the + * activation or deactivation of an entity. In this + * respect, if we execute bfq_lookup_next_entity(sd) + * in this very moment, it may, although with low + * probability, yield a different entity than that + * pointed to by sd->next_in_service. This rare event + * happens in case there was no CLASS_IDLE entity to + * serve for sd when bfq_lookup_next_entity(sd) was + * invoked for the last time, while there is now one + * such entity. + * + * If the above event happens, then the scheduling of + * such entity in CLASS_IDLE is postponed until the + * service of the sd->next_in_service entity + * finishes. In fact, when the latter is expired, + * bfq_lookup_next_entity(sd) gets called again, + * exactly to update sd->next_in_service. + */ + + /* Make next_in_service entity become in_service_entity */ + entity = sd->next_in_service; + sd->in_service_entity = entity; + + /* + * Reset the accumulator of the amount of service that + * the entity is about to receive. + */ + entity->service = 0; + + /* + * If entity is no longer a candidate for next + * service, then it must be extracted from its active + * tree, so as to make sure that it won't be + * considered when computing next_in_service. See the + * comments on the function + * bfq_no_longer_next_in_service() for details. + */ + if (bfq_no_longer_next_in_service(entity)) + bfq_active_extract(bfq_entity_service_tree(entity), + entity); + + /* + * Even if entity is not to be extracted according to + * the above check, a descendant entity may get + * extracted in one of the next iterations of this + * loop. Such an event could cause a change in + * next_in_service for the level of the descendant + * entity, and thus possibly back to this level. + * + * However, we cannot perform the resulting needed + * update of next_in_service for this level before the + * end of the whole loop, because, to know which is + * the correct next-to-serve candidate entity for each + * level, we need first to find the leaf entity to set + * in service. In fact, only after we know which is + * the next-to-serve leaf entity, we can discover + * whether the parent entity of the leaf entity + * becomes the next-to-serve, and so on. + */ + + /* Log some information */ + bfqq = bfq_entity_to_bfqq(entity); + if (bfqq) + bfq_log_bfqq(bfqd, bfqq, + "get_next_queue: this queue, finish %llu", + (((entity->finish>>10)*1000)>>10)>>2); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg(bfqd, bfqg, + "get_next_queue: this entity, finish %llu", + (((entity->finish>>10)*1000)>>10)>>2); + } +#endif + + } + + BUG_ON(!entity); + bfqq = bfq_entity_to_bfqq(entity); + BUG_ON(!bfqq); + + /* + * We can finally update all next-to-serve entities along the + * path from the leaf entity just set in service to the root. + */ + for_each_entity(entity) { + struct bfq_sched_data *sd = entity->sched_data; + + if (!bfq_update_next_in_service(sd, NULL, false)) + break; + } + + return bfqq; +} + +static void __bfq_bfqd_reset_in_service(struct bfq_data *bfqd) +{ + struct bfq_queue *in_serv_bfqq = bfqd->in_service_queue; + struct bfq_entity *in_serv_entity = &in_serv_bfqq->entity; + struct bfq_entity *entity = in_serv_entity; + +#ifndef BFQ_MQ + if (bfqd->in_service_bic) { + put_io_context(bfqd->in_service_bic->icq.ioc); + bfqd->in_service_bic = NULL; + } +#endif + + bfq_clear_bfqq_wait_request(in_serv_bfqq); + hrtimer_try_to_cancel(&bfqd->idle_slice_timer); + bfqd->in_service_queue = NULL; + + /* + * When this function is called, all in-service entities have + * been properly deactivated or requeued, so we can safely + * execute the final step: reset in_service_entity along the + * path from entity to the root. + */ + for_each_entity(entity) + entity->sched_data->in_service_entity = NULL; + + /* + * in_serv_entity is no longer in service, so, if it is in no + * service tree either, then release the service reference to + * the queue it represents (taken with bfq_get_entity). + */ + if (!in_serv_entity->on_st) + bfq_put_queue(in_serv_bfqq); +} + +static void bfq_deactivate_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq, + bool ins_into_idle_tree, bool expiration) +{ + struct bfq_entity *entity = &bfqq->entity; + + bfq_deactivate_entity(entity, ins_into_idle_tree, expiration); +} + +static void bfq_activate_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + struct bfq_service_tree *st = bfq_entity_service_tree(entity); + + BUG_ON(bfqq == bfqd->in_service_queue); + BUG_ON(entity->tree != &st->active && entity->tree != &st->idle && + entity->on_st); + + bfq_activate_requeue_entity(entity, bfq_bfqq_non_blocking_wait_rq(bfqq), + false, false); + bfq_clear_bfqq_non_blocking_wait_rq(bfqq); +} + +static void bfq_requeue_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq, + bool expiration) +{ + struct bfq_entity *entity = &bfqq->entity; + + bfq_activate_requeue_entity(entity, false, + bfqq == bfqd->in_service_queue, expiration); +} + +static void bfqg_stats_update_dequeue(struct bfq_group *bfqg); + +/* + * Called when the bfqq no longer has requests pending, remove it from + * the service tree. As a special case, it can be invoked during an + * expiration. + */ +static void bfq_del_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq, + bool expiration) +{ + BUG_ON(!bfq_bfqq_busy(bfqq)); + BUG_ON(!RB_EMPTY_ROOT(&bfqq->sort_list)); + + bfq_log_bfqq(bfqd, bfqq, "del from busy"); + + bfq_clear_bfqq_busy(bfqq); + + BUG_ON(bfqd->busy_queues == 0); + bfqd->busy_queues--; + + if (!bfqq->dispatched) + bfq_weights_tree_remove(bfqd, &bfqq->entity, + &bfqd->queue_weights_tree); + + if (bfqq->wr_coeff > 1) { + bfqd->wr_busy_queues--; + BUG_ON(bfqd->wr_busy_queues < 0); + } + + bfqg_stats_update_dequeue(bfqq_group(bfqq)); + + BUG_ON(bfqq->entity.budget < 0); + + bfq_deactivate_bfqq(bfqd, bfqq, true, expiration); +} + +/* + * Called when an inactive queue receives a new request. + */ +static void bfq_add_bfqq_busy(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + BUG_ON(bfq_bfqq_busy(bfqq)); + BUG_ON(bfqq == bfqd->in_service_queue); + + bfq_log_bfqq(bfqd, bfqq, "add to busy"); + + bfq_activate_bfqq(bfqd, bfqq); + + bfq_mark_bfqq_busy(bfqq); + bfqd->busy_queues++; + + if (!bfqq->dispatched) + if (bfqq->wr_coeff == 1) + bfq_weights_tree_add(bfqd, &bfqq->entity, + &bfqd->queue_weights_tree); + + if (bfqq->wr_coeff > 1) { + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + } + +} diff --git b/block/bfq-sq-iosched.c b/block/bfq-sq-iosched.c new file mode 100644 index 0000000..5b456e4 --- /dev/null +++ b/block/bfq-sq-iosched.c @@ -0,0 +1,5405 @@ +/* + * Budget Fair Queueing (BFQ) I/O scheduler. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2015 Paolo Valente + * + * Copyright (C) 2017 Paolo Valente + * + * Licensed under the GPL-2 as detailed in the accompanying COPYING.BFQ + * file. + * + * BFQ is a proportional-share I/O scheduler, with some extra + * low-latency capabilities. BFQ also supports full hierarchical + * scheduling through cgroups. Next paragraphs provide an introduction + * on BFQ inner workings. Details on BFQ benefits and usage can be + * found in Documentation/block/bfq-iosched.txt. + * + * BFQ is a proportional-share storage-I/O scheduling algorithm based + * on the slice-by-slice service scheme of CFQ. But BFQ assigns + * budgets, measured in number of sectors, to processes instead of + * time slices. The device is not granted to the in-service process + * for a given time slice, but until it has exhausted its assigned + * budget. This change from the time to the service domain enables BFQ + * to distribute the device throughput among processes as desired, + * without any distortion due to throughput fluctuations, or to device + * internal queueing. BFQ uses an ad hoc internal scheduler, called + * B-WF2Q+, to schedule processes according to their budgets. More + * precisely, BFQ schedules queues associated with processes. Thanks to + * the accurate policy of B-WF2Q+, BFQ can afford to assign high + * budgets to I/O-bound processes issuing sequential requests (to + * boost the throughput), and yet guarantee a low latency to + * interactive and soft real-time applications. + * + * NOTE: if the main or only goal, with a given device, is to achieve + * the maximum-possible throughput at all times, then do switch off + * all low-latency heuristics for that device, by setting low_latency + * to 0. + * + * BFQ is described in [1], where also a reference to the initial, more + * theoretical paper on BFQ can be found. The interested reader can find + * in the latter paper full details on the main algorithm, as well as + * formulas of the guarantees and formal proofs of all the properties. + * With respect to the version of BFQ presented in these papers, this + * implementation adds a few more heuristics, such as the one that + * guarantees a low latency to soft real-time applications, and a + * hierarchical extension based on H-WF2Q+. + * + * B-WF2Q+ is based on WF2Q+, that is described in [2], together with + * H-WF2Q+, while the augmented tree used to implement B-WF2Q+ with O(log N) + * complexity derives from the one introduced with EEVDF in [3]. + * + * [1] P. Valente, A. Avanzini, "Evolution of the BFQ Storage I/O + * Scheduler", Proceedings of the First Workshop on Mobile System + * Technologies (MST-2015), May 2015. + * http://algogroup.unimore.it/people/paolo/disk_sched/mst-2015.pdf + * + * http://algogroup.unimo.it/people/paolo/disk_sched/bf1-v1-suite-results.pdf + * + * [2] Jon C.R. Bennett and H. Zhang, ``Hierarchical Packet Fair Queueing + * Algorithms,'' IEEE/ACM Transactions on Networking, 5(5):675-689, + * Oct 1997. + * + * http://www.cs.cmu.edu/~hzhang/papers/TON-97-Oct.ps.gz + * + * [3] I. Stoica and H. Abdel-Wahab, ``Earliest Eligible Virtual Deadline + * First: A Flexible and Accurate Mechanism for Proportional Share + * Resource Allocation,'' technical report. + * + * http://www.cs.berkeley.edu/~istoica/papers/eevdf-tr-95.pdf + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include "blk.h" +#include "bfq.h" + +/* Expiration time of sync (0) and async (1) requests, in ns. */ +static const u64 bfq_fifo_expire[2] = { NSEC_PER_SEC / 4, NSEC_PER_SEC / 8 }; + +/* Maximum backwards seek, in KiB. */ +static const int bfq_back_max = (16 * 1024); + +/* Penalty of a backwards seek, in number of sectors. */ +static const int bfq_back_penalty = 2; + +/* Idling period duration, in ns. */ +static u32 bfq_slice_idle = (NSEC_PER_SEC / 125); + +/* Minimum number of assigned budgets for which stats are safe to compute. */ +static const int bfq_stats_min_budgets = 194; + +/* Default maximum budget values, in sectors and number of requests. */ +static const int bfq_default_max_budget = (16 * 1024); + +/* + * Async to sync throughput distribution is controlled as follows: + * when an async request is served, the entity is charged the number + * of sectors of the request, multiplied by the factor below + */ +static const int bfq_async_charge_factor = 10; + +/* Default timeout values, in jiffies, approximating CFQ defaults. */ +static const int bfq_timeout = (HZ / 8); + +static struct kmem_cache *bfq_pool; + +/* Below this threshold (in ns), we consider thinktime immediate. */ +#define BFQ_MIN_TT (2 * NSEC_PER_MSEC) + +/* hw_tag detection: parallel requests threshold and min samples needed. */ +#define BFQ_HW_QUEUE_THRESHOLD 4 +#define BFQ_HW_QUEUE_SAMPLES 32 + +#define BFQQ_SEEK_THR (sector_t)(8 * 100) +#define BFQQ_SECT_THR_NONROT (sector_t)(2 * 32) +#define BFQQ_CLOSE_THR (sector_t)(8 * 1024) +#define BFQQ_SEEKY(bfqq) (hweight32(bfqq->seek_history) > 32/8) + +/* Min number of samples required to perform peak-rate update */ +#define BFQ_RATE_MIN_SAMPLES 32 +/* Min observation time interval required to perform a peak-rate update (ns) */ +#define BFQ_RATE_MIN_INTERVAL (300*NSEC_PER_MSEC) +/* Target observation time interval for a peak-rate update (ns) */ +#define BFQ_RATE_REF_INTERVAL NSEC_PER_SEC + +/* Shift used for peak rate fixed precision calculations. */ +#define BFQ_RATE_SHIFT 16 + +/* + * By default, BFQ computes the duration of the weight raising for + * interactive applications automatically, using the following formula: + * duration = (R / r) * T, where r is the peak rate of the device, and + * R and T are two reference parameters. + * In particular, R is the peak rate of the reference device (see below), + * and T is a reference time: given the systems that are likely to be + * installed on the reference device according to its speed class, T is + * about the maximum time needed, under BFQ and while reading two files in + * parallel, to load typical large applications on these systems. + * In practice, the slower/faster the device at hand is, the more/less it + * takes to load applications with respect to the reference device. + * Accordingly, the longer/shorter BFQ grants weight raising to interactive + * applications. + * + * BFQ uses four different reference pairs (R, T), depending on: + * . whether the device is rotational or non-rotational; + * . whether the device is slow, such as old or portable HDDs, as well as + * SD cards, or fast, such as newer HDDs and SSDs. + * + * The device's speed class is dynamically (re)detected in + * bfq_update_peak_rate() every time the estimated peak rate is updated. + * + * In the following definitions, R_slow[0]/R_fast[0] and + * T_slow[0]/T_fast[0] are the reference values for a slow/fast + * rotational device, whereas R_slow[1]/R_fast[1] and + * T_slow[1]/T_fast[1] are the reference values for a slow/fast + * non-rotational device. Finally, device_speed_thresh are the + * thresholds used to switch between speed classes. The reference + * rates are not the actual peak rates of the devices used as a + * reference, but slightly lower values. The reason for using these + * slightly lower values is that the peak-rate estimator tends to + * yield slightly lower values than the actual peak rate (it can yield + * the actual peak rate only if there is only one process doing I/O, + * and the process does sequential I/O). + * + * Both the reference peak rates and the thresholds are measured in + * sectors/usec, left-shifted by BFQ_RATE_SHIFT. + */ +static int R_slow[2] = {1000, 10700}; +static int R_fast[2] = {14000, 33000}; +/* + * To improve readability, a conversion function is used to initialize the + * following arrays, which entails that they can be initialized only in a + * function. + */ +static int T_slow[2]; +static int T_fast[2]; +static int device_speed_thresh[2]; + +#define BFQ_SERVICE_TREE_INIT ((struct bfq_service_tree) \ + { RB_ROOT, RB_ROOT, NULL, NULL, 0, 0 }) + +#define RQ_BIC(rq) ((struct bfq_io_cq *) (rq)->elv.priv[0]) +#define RQ_BFQQ(rq) ((rq)->elv.priv[1]) + +static void bfq_schedule_dispatch(struct bfq_data *bfqd); + +#include "bfq-ioc.c" +#include "bfq-sched.c" +#include "bfq-cgroup-included.c" + +#define bfq_class_idle(bfqq) ((bfqq)->ioprio_class == IOPRIO_CLASS_IDLE) +#define bfq_class_rt(bfqq) ((bfqq)->ioprio_class == IOPRIO_CLASS_RT) + +#define bfq_sample_valid(samples) ((samples) > 80) + +/* + * Scheduler run of queue, if there are requests pending and no one in the + * driver that will restart queueing. + */ +static void bfq_schedule_dispatch(struct bfq_data *bfqd) +{ + if (bfqd->queued != 0) { + bfq_log(bfqd, "schedule dispatch"); + kblockd_schedule_work(&bfqd->unplug_work); + } +} + +/* + * Lifted from AS - choose which of rq1 and rq2 that is best served now. + * We choose the request that is closesr to the head right now. Distance + * behind the head is penalized and only allowed to a certain extent. + */ +static struct request *bfq_choose_req(struct bfq_data *bfqd, + struct request *rq1, + struct request *rq2, + sector_t last) +{ + sector_t s1, s2, d1 = 0, d2 = 0; + unsigned long back_max; +#define BFQ_RQ1_WRAP 0x01 /* request 1 wraps */ +#define BFQ_RQ2_WRAP 0x02 /* request 2 wraps */ + unsigned int wrap = 0; /* bit mask: requests behind the disk head? */ + + if (!rq1 || rq1 == rq2) + return rq2; + if (!rq2) + return rq1; + + if (rq_is_sync(rq1) && !rq_is_sync(rq2)) + return rq1; + else if (rq_is_sync(rq2) && !rq_is_sync(rq1)) + return rq2; + if ((rq1->cmd_flags & REQ_META) && !(rq2->cmd_flags & REQ_META)) + return rq1; + else if ((rq2->cmd_flags & REQ_META) && !(rq1->cmd_flags & REQ_META)) + return rq2; + + s1 = blk_rq_pos(rq1); + s2 = blk_rq_pos(rq2); + + /* + * By definition, 1KiB is 2 sectors. + */ + back_max = bfqd->bfq_back_max * 2; + + /* + * Strict one way elevator _except_ in the case where we allow + * short backward seeks which are biased as twice the cost of a + * similar forward seek. + */ + if (s1 >= last) + d1 = s1 - last; + else if (s1 + back_max >= last) + d1 = (last - s1) * bfqd->bfq_back_penalty; + else + wrap |= BFQ_RQ1_WRAP; + + if (s2 >= last) + d2 = s2 - last; + else if (s2 + back_max >= last) + d2 = (last - s2) * bfqd->bfq_back_penalty; + else + wrap |= BFQ_RQ2_WRAP; + + /* Found required data */ + + /* + * By doing switch() on the bit mask "wrap" we avoid having to + * check two variables for all permutations: --> faster! + */ + switch (wrap) { + case 0: /* common case for CFQ: rq1 and rq2 not wrapped */ + if (d1 < d2) + return rq1; + else if (d2 < d1) + return rq2; + + if (s1 >= s2) + return rq1; + else + return rq2; + + case BFQ_RQ2_WRAP: + return rq1; + case BFQ_RQ1_WRAP: + return rq2; + case (BFQ_RQ1_WRAP|BFQ_RQ2_WRAP): /* both rqs wrapped */ + default: + /* + * Since both rqs are wrapped, + * start with the one that's further behind head + * (--> only *one* back seek required), + * since back seek takes more time than forward. + */ + if (s1 <= s2) + return rq1; + else + return rq2; + } +} + +static struct bfq_queue * +bfq_rq_pos_tree_lookup(struct bfq_data *bfqd, struct rb_root *root, + sector_t sector, struct rb_node **ret_parent, + struct rb_node ***rb_link) +{ + struct rb_node **p, *parent; + struct bfq_queue *bfqq = NULL; + + parent = NULL; + p = &root->rb_node; + while (*p) { + struct rb_node **n; + + parent = *p; + bfqq = rb_entry(parent, struct bfq_queue, pos_node); + + /* + * Sort strictly based on sector. Smallest to the left, + * largest to the right. + */ + if (sector > blk_rq_pos(bfqq->next_rq)) + n = &(*p)->rb_right; + else if (sector < blk_rq_pos(bfqq->next_rq)) + n = &(*p)->rb_left; + else + break; + p = n; + bfqq = NULL; + } + + *ret_parent = parent; + if (rb_link) + *rb_link = p; + + bfq_log(bfqd, "rq_pos_tree_lookup %llu: returning %d", + (unsigned long long) sector, + bfqq ? bfqq->pid : 0); + + return bfqq; +} + +static void bfq_pos_tree_add_move(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct rb_node **p, *parent; + struct bfq_queue *__bfqq; + + if (bfqq->pos_root) { + rb_erase(&bfqq->pos_node, bfqq->pos_root); + bfqq->pos_root = NULL; + } + + if (bfq_class_idle(bfqq)) + return; + if (!bfqq->next_rq) + return; + + bfqq->pos_root = &bfq_bfqq_to_bfqg(bfqq)->rq_pos_tree; + __bfqq = bfq_rq_pos_tree_lookup(bfqd, bfqq->pos_root, + blk_rq_pos(bfqq->next_rq), &parent, &p); + if (!__bfqq) { + rb_link_node(&bfqq->pos_node, parent, p); + rb_insert_color(&bfqq->pos_node, bfqq->pos_root); + } else + bfqq->pos_root = NULL; +} + +/* + * Tell whether there are active queues or groups with differentiated weights. + */ +static bool bfq_differentiated_weights(struct bfq_data *bfqd) +{ + /* + * For weights to differ, at least one of the trees must contain + * at least two nodes. + */ + return (!RB_EMPTY_ROOT(&bfqd->queue_weights_tree) && + (bfqd->queue_weights_tree.rb_node->rb_left || + bfqd->queue_weights_tree.rb_node->rb_right) +#ifdef BFQ_GROUP_IOSCHED_ENABLED + ) || + (!RB_EMPTY_ROOT(&bfqd->group_weights_tree) && + (bfqd->group_weights_tree.rb_node->rb_left || + bfqd->group_weights_tree.rb_node->rb_right) +#endif + ); +} + +/* + * The following function returns true if every queue must receive the + * same share of the throughput (this condition is used when deciding + * whether idling may be disabled, see the comments in the function + * bfq_bfqq_may_idle()). + * + * Such a scenario occurs when: + * 1) all active queues have the same weight, + * 2) all active groups at the same level in the groups tree have the same + * weight, + * 3) all active groups at the same level in the groups tree have the same + * number of children. + * + * Unfortunately, keeping the necessary state for evaluating exactly the + * above symmetry conditions would be quite complex and time-consuming. + * Therefore this function evaluates, instead, the following stronger + * sub-conditions, for which it is much easier to maintain the needed + * state: + * 1) all active queues have the same weight, + * 2) all active groups have the same weight, + * 3) all active groups have at most one active child each. + * In particular, the last two conditions are always true if hierarchical + * support and the cgroups interface are not enabled, thus no state needs + * to be maintained in this case. + */ +static bool bfq_symmetric_scenario(struct bfq_data *bfqd) +{ + return !bfq_differentiated_weights(bfqd); +} + +/* + * If the weight-counter tree passed as input contains no counter for + * the weight of the input entity, then add that counter; otherwise just + * increment the existing counter. + * + * Note that weight-counter trees contain few nodes in mostly symmetric + * scenarios. For example, if all queues have the same weight, then the + * weight-counter tree for the queues may contain at most one node. + * This holds even if low_latency is on, because weight-raised queues + * are not inserted in the tree. + * In most scenarios, the rate at which nodes are created/destroyed + * should be low too. + */ +static void bfq_weights_tree_add(struct bfq_data *bfqd, + struct bfq_entity *entity, + struct rb_root *root) +{ + struct rb_node **new = &(root->rb_node), *parent = NULL; + + /* + * Do not insert if the entity is already associated with a + * counter, which happens if: + * 1) the entity is associated with a queue, + * 2) a request arrival has caused the queue to become both + * non-weight-raised, and hence change its weight, and + * backlogged; in this respect, each of the two events + * causes an invocation of this function, + * 3) this is the invocation of this function caused by the + * second event. This second invocation is actually useless, + * and we handle this fact by exiting immediately. More + * efficient or clearer solutions might possibly be adopted. + */ + if (entity->weight_counter) + return; + + while (*new) { + struct bfq_weight_counter *__counter = container_of(*new, + struct bfq_weight_counter, + weights_node); + parent = *new; + + if (entity->weight == __counter->weight) { + entity->weight_counter = __counter; + goto inc_counter; + } + if (entity->weight < __counter->weight) + new = &((*new)->rb_left); + else + new = &((*new)->rb_right); + } + + entity->weight_counter = kzalloc(sizeof(struct bfq_weight_counter), + GFP_ATOMIC); + + /* + * In the unlucky event of an allocation failure, we just + * exit. This will cause the weight of entity to not be + * considered in bfq_differentiated_weights, which, in its + * turn, causes the scenario to be deemed wrongly symmetric in + * case entity's weight would have been the only weight making + * the scenario asymmetric. On the bright side, no unbalance + * will however occur when entity becomes inactive again (the + * invocation of this function is triggered by an activation + * of entity). In fact, bfq_weights_tree_remove does nothing + * if !entity->weight_counter. + */ + if (unlikely(!entity->weight_counter)) + return; + + entity->weight_counter->weight = entity->weight; + rb_link_node(&entity->weight_counter->weights_node, parent, new); + rb_insert_color(&entity->weight_counter->weights_node, root); + +inc_counter: + entity->weight_counter->num_active++; +} + +/* + * Decrement the weight counter associated with the entity, and, if the + * counter reaches 0, remove the counter from the tree. + * See the comments to the function bfq_weights_tree_add() for considerations + * about overhead. + */ +static void bfq_weights_tree_remove(struct bfq_data *bfqd, + struct bfq_entity *entity, + struct rb_root *root) +{ + if (!entity->weight_counter) + return; + + BUG_ON(RB_EMPTY_ROOT(root)); + BUG_ON(entity->weight_counter->weight != entity->weight); + + BUG_ON(!entity->weight_counter->num_active); + entity->weight_counter->num_active--; + if (entity->weight_counter->num_active > 0) + goto reset_entity_pointer; + + rb_erase(&entity->weight_counter->weights_node, root); + kfree(entity->weight_counter); + +reset_entity_pointer: + entity->weight_counter = NULL; +} + +/* + * Return expired entry, or NULL to just start from scratch in rbtree. + */ +static struct request *bfq_check_fifo(struct bfq_queue *bfqq, + struct request *last) +{ + struct request *rq; + + if (bfq_bfqq_fifo_expire(bfqq)) + return NULL; + + bfq_mark_bfqq_fifo_expire(bfqq); + + rq = rq_entry_fifo(bfqq->fifo.next); + + if (rq == last || ktime_get_ns() < rq->fifo_time) + return NULL; + + bfq_log_bfqq(bfqq->bfqd, bfqq, "check_fifo: returned %p", rq); + BUG_ON(RB_EMPTY_NODE(&rq->rb_node)); + return rq; +} + +static struct request *bfq_find_next_rq(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + struct request *last) +{ + struct rb_node *rbnext = rb_next(&last->rb_node); + struct rb_node *rbprev = rb_prev(&last->rb_node); + struct request *next, *prev = NULL; + + BUG_ON(list_empty(&bfqq->fifo)); + + /* Follow expired path, else get first next available. */ + next = bfq_check_fifo(bfqq, last); + if (next) { + BUG_ON(next == last); + return next; + } + + BUG_ON(RB_EMPTY_NODE(&last->rb_node)); + + if (rbprev) + prev = rb_entry_rq(rbprev); + + if (rbnext) + next = rb_entry_rq(rbnext); + else { + rbnext = rb_first(&bfqq->sort_list); + if (rbnext && rbnext != &last->rb_node) + next = rb_entry_rq(rbnext); + } + + return bfq_choose_req(bfqd, next, prev, blk_rq_pos(last)); +} + +/* see the definition of bfq_async_charge_factor for details */ +static unsigned long bfq_serv_to_charge(struct request *rq, + struct bfq_queue *bfqq) +{ + if (bfq_bfqq_sync(bfqq) || bfqq->wr_coeff > 1) + return blk_rq_sectors(rq); + + /* + * If there are no weight-raised queues, then amplify service + * by just the async charge factor; otherwise amplify service + * by twice the async charge factor, to further reduce latency + * for weight-raised queues. + */ + if (bfqq->bfqd->wr_busy_queues == 0) + return blk_rq_sectors(rq) * bfq_async_charge_factor; + + return blk_rq_sectors(rq) * 2 * bfq_async_charge_factor; +} + +/** + * bfq_updated_next_req - update the queue after a new next_rq selection. + * @bfqd: the device data the queue belongs to. + * @bfqq: the queue to update. + * + * If the first request of a queue changes we make sure that the queue + * has enough budget to serve at least its first request (if the + * request has grown). We do this because if the queue has not enough + * budget for its first request, it has to go through two dispatch + * rounds to actually get it dispatched. + */ +static void bfq_updated_next_req(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + struct bfq_service_tree *st = bfq_entity_service_tree(entity); + struct request *next_rq = bfqq->next_rq; + unsigned long new_budget; + + if (!next_rq) + return; + + if (bfqq == bfqd->in_service_queue) + /* + * In order not to break guarantees, budgets cannot be + * changed after an entity has been selected. + */ + return; + + BUG_ON(entity->tree != &st->active); + BUG_ON(entity == entity->sched_data->in_service_entity); + + new_budget = max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(next_rq, bfqq)); + if (entity->budget != new_budget) { + entity->budget = new_budget; + bfq_log_bfqq(bfqd, bfqq, "updated next rq: new budget %lu", + new_budget); + bfq_requeue_bfqq(bfqd, bfqq, false); + } +} + +static unsigned int bfq_wr_duration(struct bfq_data *bfqd) +{ + u64 dur; + + if (bfqd->bfq_wr_max_time > 0) + return bfqd->bfq_wr_max_time; + + dur = bfqd->RT_prod; + do_div(dur, bfqd->peak_rate); + + /* + * Limit duration between 3 and 13 seconds. Tests show that + * higher values than 13 seconds often yield the opposite of + * the desired result, i.e., worsen responsiveness by letting + * non-interactive and non-soft-real-time applications + * preserve weight raising for a too long time interval. + * + * On the other end, lower values than 3 seconds make it + * difficult for most interactive tasks to complete their jobs + * before weight-raising finishes. + */ + if (dur > msecs_to_jiffies(13000)) + dur = msecs_to_jiffies(13000); + else if (dur < msecs_to_jiffies(3000)) + dur = msecs_to_jiffies(3000); + + return dur; +} + +static void +bfq_bfqq_resume_state(struct bfq_queue *bfqq, struct bfq_data *bfqd, + struct bfq_io_cq *bic, bool bfq_already_existing) +{ + unsigned int old_wr_coeff; + bool busy = bfq_already_existing && bfq_bfqq_busy(bfqq); + + if (bic->saved_has_short_ttime) + bfq_mark_bfqq_has_short_ttime(bfqq); + else + bfq_clear_bfqq_has_short_ttime(bfqq); + + if (bic->saved_IO_bound) + bfq_mark_bfqq_IO_bound(bfqq); + else + bfq_clear_bfqq_IO_bound(bfqq); + + if (unlikely(busy)) + old_wr_coeff = bfqq->wr_coeff; + + bfqq->wr_coeff = bic->saved_wr_coeff; + bfqq->wr_start_at_switch_to_srt = bic->saved_wr_start_at_switch_to_srt; + BUG_ON(time_is_after_jiffies(bfqq->wr_start_at_switch_to_srt)); + bfqq->last_wr_start_finish = bic->saved_last_wr_start_finish; + bfqq->wr_cur_max_time = bic->saved_wr_cur_max_time; + BUG_ON(time_is_after_jiffies(bfqq->last_wr_start_finish)); + + if (bfqq->wr_coeff > 1 && (bfq_bfqq_in_large_burst(bfqq) || + time_is_before_jiffies(bfqq->last_wr_start_finish + + bfqq->wr_cur_max_time))) { + bfq_log_bfqq(bfqq->bfqd, bfqq, + "resume state: switching off wr (%lu + %lu < %lu)", + bfqq->last_wr_start_finish, bfqq->wr_cur_max_time, + jiffies); + + bfqq->wr_coeff = 1; + } + + /* make sure weight will be updated, however we got here */ + bfqq->entity.prio_changed = 1; + + if (likely(!busy)) + return; + + if (old_wr_coeff == 1 && bfqq->wr_coeff > 1) { + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + } else if (old_wr_coeff > 1 && bfqq->wr_coeff == 1) { + bfqd->wr_busy_queues--; + BUG_ON(bfqd->wr_busy_queues < 0); + } +} + +static int bfqq_process_refs(struct bfq_queue *bfqq) +{ + int process_refs, io_refs; + + lockdep_assert_held(bfqq->bfqd->queue->queue_lock); + + io_refs = bfqq->allocated[READ] + bfqq->allocated[WRITE]; + process_refs = bfqq->ref - io_refs - bfqq->entity.on_st; + BUG_ON(process_refs < 0); + return process_refs; +} + +/* Empty burst list and add just bfqq (see comments to bfq_handle_burst) */ +static void bfq_reset_burst_list(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct bfq_queue *item; + struct hlist_node *n; + + hlist_for_each_entry_safe(item, n, &bfqd->burst_list, burst_list_node) + hlist_del_init(&item->burst_list_node); + hlist_add_head(&bfqq->burst_list_node, &bfqd->burst_list); + bfqd->burst_size = 1; + bfqd->burst_parent_entity = bfqq->entity.parent; +} + +/* Add bfqq to the list of queues in current burst (see bfq_handle_burst) */ +static void bfq_add_to_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + /* Increment burst size to take into account also bfqq */ + bfqd->burst_size++; + + bfq_log_bfqq(bfqd, bfqq, "add_to_burst %d", bfqd->burst_size); + + BUG_ON(bfqd->burst_size > bfqd->bfq_large_burst_thresh); + + if (bfqd->burst_size == bfqd->bfq_large_burst_thresh) { + struct bfq_queue *pos, *bfqq_item; + struct hlist_node *n; + + /* + * Enough queues have been activated shortly after each + * other to consider this burst as large. + */ + bfqd->large_burst = true; + bfq_log_bfqq(bfqd, bfqq, "add_to_burst: large burst started"); + + /* + * We can now mark all queues in the burst list as + * belonging to a large burst. + */ + hlist_for_each_entry(bfqq_item, &bfqd->burst_list, + burst_list_node) { + bfq_mark_bfqq_in_large_burst(bfqq_item); + bfq_log_bfqq(bfqd, bfqq_item, "marked in large burst"); + } + bfq_mark_bfqq_in_large_burst(bfqq); + bfq_log_bfqq(bfqd, bfqq, "marked in large burst"); + + /* + * From now on, and until the current burst finishes, any + * new queue being activated shortly after the last queue + * was inserted in the burst can be immediately marked as + * belonging to a large burst. So the burst list is not + * needed any more. Remove it. + */ + hlist_for_each_entry_safe(pos, n, &bfqd->burst_list, + burst_list_node) + hlist_del_init(&pos->burst_list_node); + } else /* + * Burst not yet large: add bfqq to the burst list. Do + * not increment the ref counter for bfqq, because bfqq + * is removed from the burst list before freeing bfqq + * in put_queue. + */ + hlist_add_head(&bfqq->burst_list_node, &bfqd->burst_list); +} + +/* + * If many queues belonging to the same group happen to be created + * shortly after each other, then the processes associated with these + * queues have typically a common goal. In particular, bursts of queue + * creations are usually caused by services or applications that spawn + * many parallel threads/processes. Examples are systemd during boot, + * or git grep. To help these processes get their job done as soon as + * possible, it is usually better to not grant either weight-raising + * or device idling to their queues. + * + * In this comment we describe, firstly, the reasons why this fact + * holds, and, secondly, the next function, which implements the main + * steps needed to properly mark these queues so that they can then be + * treated in a different way. + * + * The above services or applications benefit mostly from a high + * throughput: the quicker the requests of the activated queues are + * cumulatively served, the sooner the target job of these queues gets + * completed. As a consequence, weight-raising any of these queues, + * which also implies idling the device for it, is almost always + * counterproductive. In most cases it just lowers throughput. + * + * On the other hand, a burst of queue creations may be caused also by + * the start of an application that does not consist of a lot of + * parallel I/O-bound threads. In fact, with a complex application, + * several short processes may need to be executed to start-up the + * application. In this respect, to start an application as quickly as + * possible, the best thing to do is in any case to privilege the I/O + * related to the application with respect to all other + * I/O. Therefore, the best strategy to start as quickly as possible + * an application that causes a burst of queue creations is to + * weight-raise all the queues created during the burst. This is the + * exact opposite of the best strategy for the other type of bursts. + * + * In the end, to take the best action for each of the two cases, the + * two types of bursts need to be distinguished. Fortunately, this + * seems relatively easy, by looking at the sizes of the bursts. In + * particular, we found a threshold such that only bursts with a + * larger size than that threshold are apparently caused by + * services or commands such as systemd or git grep. For brevity, + * hereafter we call just 'large' these bursts. BFQ *does not* + * weight-raise queues whose creation occurs in a large burst. In + * addition, for each of these queues BFQ performs or does not perform + * idling depending on which choice boosts the throughput more. The + * exact choice depends on the device and request pattern at + * hand. + * + * Unfortunately, false positives may occur while an interactive task + * is starting (e.g., an application is being started). The + * consequence is that the queues associated with the task do not + * enjoy weight raising as expected. Fortunately these false positives + * are very rare. They typically occur if some service happens to + * start doing I/O exactly when the interactive task starts. + * + * Turning back to the next function, it implements all the steps + * needed to detect the occurrence of a large burst and to properly + * mark all the queues belonging to it (so that they can then be + * treated in a different way). This goal is achieved by maintaining a + * "burst list" that holds, temporarily, the queues that belong to the + * burst in progress. The list is then used to mark these queues as + * belonging to a large burst if the burst does become large. The main + * steps are the following. + * + * . when the very first queue is created, the queue is inserted into the + * list (as it could be the first queue in a possible burst) + * + * . if the current burst has not yet become large, and a queue Q that does + * not yet belong to the burst is activated shortly after the last time + * at which a new queue entered the burst list, then the function appends + * Q to the burst list + * + * . if, as a consequence of the previous step, the burst size reaches + * the large-burst threshold, then + * + * . all the queues in the burst list are marked as belonging to a + * large burst + * + * . the burst list is deleted; in fact, the burst list already served + * its purpose (keeping temporarily track of the queues in a burst, + * so as to be able to mark them as belonging to a large burst in the + * previous sub-step), and now is not needed any more + * + * . the device enters a large-burst mode + * + * . if a queue Q that does not belong to the burst is created while + * the device is in large-burst mode and shortly after the last time + * at which a queue either entered the burst list or was marked as + * belonging to the current large burst, then Q is immediately marked + * as belonging to a large burst. + * + * . if a queue Q that does not belong to the burst is created a while + * later, i.e., not shortly after, than the last time at which a queue + * either entered the burst list or was marked as belonging to the + * current large burst, then the current burst is deemed as finished and: + * + * . the large-burst mode is reset if set + * + * . the burst list is emptied + * + * . Q is inserted in the burst list, as Q may be the first queue + * in a possible new burst (then the burst list contains just Q + * after this step). + */ +static void bfq_handle_burst(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + /* + * If bfqq is already in the burst list or is part of a large + * burst, or finally has just been split, then there is + * nothing else to do. + */ + if (!hlist_unhashed(&bfqq->burst_list_node) || + bfq_bfqq_in_large_burst(bfqq) || + time_is_after_eq_jiffies(bfqq->split_time + + msecs_to_jiffies(10))) + return; + + /* + * If bfqq's creation happens late enough, or bfqq belongs to + * a different group than the burst group, then the current + * burst is finished, and related data structures must be + * reset. + * + * In this respect, consider the special case where bfqq is + * the very first queue created after BFQ is selected for this + * device. In this case, last_ins_in_burst and + * burst_parent_entity are not yet significant when we get + * here. But it is easy to verify that, whether or not the + * following condition is true, bfqq will end up being + * inserted into the burst list. In particular the list will + * happen to contain only bfqq. And this is exactly what has + * to happen, as bfqq may be the first queue of the first + * burst. + */ + if (time_is_before_jiffies(bfqd->last_ins_in_burst + + bfqd->bfq_burst_interval) || + bfqq->entity.parent != bfqd->burst_parent_entity) { + bfqd->large_burst = false; + bfq_reset_burst_list(bfqd, bfqq); + bfq_log_bfqq(bfqd, bfqq, + "handle_burst: late activation or different group"); + goto end; + } + + /* + * If we get here, then bfqq is being activated shortly after the + * last queue. So, if the current burst is also large, we can mark + * bfqq as belonging to this large burst immediately. + */ + if (bfqd->large_burst) { + bfq_log_bfqq(bfqd, bfqq, "handle_burst: marked in burst"); + bfq_mark_bfqq_in_large_burst(bfqq); + goto end; + } + + /* + * If we get here, then a large-burst state has not yet been + * reached, but bfqq is being activated shortly after the last + * queue. Then we add bfqq to the burst. + */ + bfq_add_to_burst(bfqd, bfqq); +end: + /* + * At this point, bfqq either has been added to the current + * burst or has caused the current burst to terminate and a + * possible new burst to start. In particular, in the second + * case, bfqq has become the first queue in the possible new + * burst. In both cases last_ins_in_burst needs to be moved + * forward. + */ + bfqd->last_ins_in_burst = jiffies; + +} + +static int bfq_bfqq_budget_left(struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + + return entity->budget - entity->service; +} + +/* + * If enough samples have been computed, return the current max budget + * stored in bfqd, which is dynamically updated according to the + * estimated disk peak rate; otherwise return the default max budget + */ +static int bfq_max_budget(struct bfq_data *bfqd) +{ + if (bfqd->budgets_assigned < bfq_stats_min_budgets) + return bfq_default_max_budget; + else + return bfqd->bfq_max_budget; +} + +/* + * Return min budget, which is a fraction of the current or default + * max budget (trying with 1/32) + */ +static int bfq_min_budget(struct bfq_data *bfqd) +{ + if (bfqd->budgets_assigned < bfq_stats_min_budgets) + return bfq_default_max_budget / 32; + else + return bfqd->bfq_max_budget / 32; +} + +static void bfq_bfqq_expire(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool compensate, + enum bfqq_expiration reason); + +/* + * The next function, invoked after the input queue bfqq switches from + * idle to busy, updates the budget of bfqq. The function also tells + * whether the in-service queue should be expired, by returning + * true. The purpose of expiring the in-service queue is to give bfqq + * the chance to possibly preempt the in-service queue, and the reason + * for preempting the in-service queue is to achieve one of the two + * goals below. + * + * 1. Guarantee to bfqq its reserved bandwidth even if bfqq has + * expired because it has remained idle. In particular, bfqq may have + * expired for one of the following two reasons: + * + * - BFQ_BFQQ_NO_MORE_REQUEST bfqq did not enjoy any device idling and + * did not make it to issue a new request before its last request + * was served; + * + * - BFQ_BFQQ_TOO_IDLE bfqq did enjoy device idling, but did not issue + * a new request before the expiration of the idling-time. + * + * Even if bfqq has expired for one of the above reasons, the process + * associated with the queue may be however issuing requests greedily, + * and thus be sensitive to the bandwidth it receives (bfqq may have + * remained idle for other reasons: CPU high load, bfqq not enjoying + * idling, I/O throttling somewhere in the path from the process to + * the I/O scheduler, ...). But if, after every expiration for one of + * the above two reasons, bfqq has to wait for the service of at least + * one full budget of another queue before being served again, then + * bfqq is likely to get a much lower bandwidth or resource time than + * its reserved ones. To address this issue, two countermeasures need + * to be taken. + * + * First, the budget and the timestamps of bfqq need to be updated in + * a special way on bfqq reactivation: they need to be updated as if + * bfqq did not remain idle and did not expire. In fact, if they are + * computed as if bfqq expired and remained idle until reactivation, + * then the process associated with bfqq is treated as if, instead of + * being greedy, it stopped issuing requests when bfqq remained idle, + * and restarts issuing requests only on this reactivation. In other + * words, the scheduler does not help the process recover the "service + * hole" between bfqq expiration and reactivation. As a consequence, + * the process receives a lower bandwidth than its reserved one. In + * contrast, to recover this hole, the budget must be updated as if + * bfqq was not expired at all before this reactivation, i.e., it must + * be set to the value of the remaining budget when bfqq was + * expired. Along the same line, timestamps need to be assigned the + * value they had the last time bfqq was selected for service, i.e., + * before last expiration. Thus timestamps need to be back-shifted + * with respect to their normal computation (see [1] for more details + * on this tricky aspect). + * + * Secondly, to allow the process to recover the hole, the in-service + * queue must be expired too, to give bfqq the chance to preempt it + * immediately. In fact, if bfqq has to wait for a full budget of the + * in-service queue to be completed, then it may become impossible to + * let the process recover the hole, even if the back-shifted + * timestamps of bfqq are lower than those of the in-service queue. If + * this happens for most or all of the holes, then the process may not + * receive its reserved bandwidth. In this respect, it is worth noting + * that, being the service of outstanding requests unpreemptible, a + * little fraction of the holes may however be unrecoverable, thereby + * causing a little loss of bandwidth. + * + * The last important point is detecting whether bfqq does need this + * bandwidth recovery. In this respect, the next function deems the + * process associated with bfqq greedy, and thus allows it to recover + * the hole, if: 1) the process is waiting for the arrival of a new + * request (which implies that bfqq expired for one of the above two + * reasons), and 2) such a request has arrived soon. The first + * condition is controlled through the flag non_blocking_wait_rq, + * while the second through the flag arrived_in_time. If both + * conditions hold, then the function computes the budget in the + * above-described special way, and signals that the in-service queue + * should be expired. Timestamp back-shifting is done later in + * __bfq_activate_entity. + * + * 2. Reduce latency. Even if timestamps are not backshifted to let + * the process associated with bfqq recover a service hole, bfqq may + * however happen to have, after being (re)activated, a lower finish + * timestamp than the in-service queue. That is, the next budget of + * bfqq may have to be completed before the one of the in-service + * queue. If this is the case, then preempting the in-service queue + * allows this goal to be achieved, apart from the unpreemptible, + * outstanding requests mentioned above. + * + * Unfortunately, regardless of which of the above two goals one wants + * to achieve, service trees need first to be updated to know whether + * the in-service queue must be preempted. To have service trees + * correctly updated, the in-service queue must be expired and + * rescheduled, and bfqq must be scheduled too. This is one of the + * most costly operations (in future versions, the scheduling + * mechanism may be re-designed in such a way to make it possible to + * know whether preemption is needed without needing to update service + * trees). In addition, queue preemptions almost always cause random + * I/O, and thus loss of throughput. Because of these facts, the next + * function adopts the following simple scheme to avoid both costly + * operations and too frequent preemptions: it requests the expiration + * of the in-service queue (unconditionally) only for queues that need + * to recover a hole, or that either are weight-raised or deserve to + * be weight-raised. + */ +static bool bfq_bfqq_update_budg_for_activation(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool arrived_in_time, + bool wr_or_deserves_wr) +{ + struct bfq_entity *entity = &bfqq->entity; + + if (bfq_bfqq_non_blocking_wait_rq(bfqq) && arrived_in_time) { + /* + * We do not clear the flag non_blocking_wait_rq here, as + * the latter is used in bfq_activate_bfqq to signal + * that timestamps need to be back-shifted (and is + * cleared right after). + */ + + /* + * In next assignment we rely on that either + * entity->service or entity->budget are not updated + * on expiration if bfqq is empty (see + * __bfq_bfqq_recalc_budget). Thus both quantities + * remain unchanged after such an expiration, and the + * following statement therefore assigns to + * entity->budget the remaining budget on such an + * expiration. For clarity, entity->service is not + * updated on expiration in any case, and, in normal + * operation, is reset only when bfqq is selected for + * service (see bfq_get_next_queue). + */ + BUG_ON(bfqq->max_budget < 0); + entity->budget = min_t(unsigned long, + bfq_bfqq_budget_left(bfqq), + bfqq->max_budget); + + BUG_ON(entity->budget < 0); + return true; + } + + BUG_ON(bfqq->max_budget < 0); + entity->budget = max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(bfqq->next_rq, bfqq)); + BUG_ON(entity->budget < 0); + + bfq_clear_bfqq_non_blocking_wait_rq(bfqq); + return wr_or_deserves_wr; +} + +/* + * Return the farthest future time instant according to jiffies + * macros. + */ +static unsigned long bfq_greatest_from_now(void) +{ + return jiffies + MAX_JIFFY_OFFSET; +} + +/* + * Return the farthest past time instant according to jiffies + * macros. + */ +static unsigned long bfq_smallest_from_now(void) +{ + return jiffies - MAX_JIFFY_OFFSET; +} + +static void bfq_update_bfqq_wr_on_rq_arrival(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + unsigned int old_wr_coeff, + bool wr_or_deserves_wr, + bool interactive, + bool in_burst, + bool soft_rt) +{ + if (old_wr_coeff == 1 && wr_or_deserves_wr) { + /* start a weight-raising period */ + if (interactive) { + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + } else { + /* + * No interactive weight raising in progress + * here: assign minus infinity to + * wr_start_at_switch_to_srt, to make sure + * that, at the end of the soft-real-time + * weight raising periods that is starting + * now, no interactive weight-raising period + * may be wrongly considered as still in + * progress (and thus actually started by + * mistake). + */ + bfqq->wr_start_at_switch_to_srt = + bfq_smallest_from_now(); + bfqq->wr_coeff = bfqd->bfq_wr_coeff * + BFQ_SOFTRT_WEIGHT_FACTOR; + bfqq->wr_cur_max_time = + bfqd->bfq_wr_rt_max_time; + } + /* + * If needed, further reduce budget to make sure it is + * close to bfqq's backlog, so as to reduce the + * scheduling-error component due to a too large + * budget. Do not care about throughput consequences, + * but only about latency. Finally, do not assign a + * too small budget either, to avoid increasing + * latency by causing too frequent expirations. + */ + bfqq->entity.budget = min_t(unsigned long, + bfqq->entity.budget, + 2 * bfq_min_budget(bfqd)); + + bfq_log_bfqq(bfqd, bfqq, + "wrais starting at %lu, rais_max_time %u", + jiffies, + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } else if (old_wr_coeff > 1) { + if (interactive) { /* update wr coeff and duration */ + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + } else if (in_burst) { + bfqq->wr_coeff = 1; + bfq_log_bfqq(bfqd, bfqq, + "wrais ending at %lu, rais_max_time %u", + jiffies, + jiffies_to_msecs(bfqq-> + wr_cur_max_time)); + } else if (soft_rt) { + /* + * The application is now or still meeting the + * requirements for being deemed soft rt. We + * can then correctly and safely (re)charge + * the weight-raising duration for the + * application with the weight-raising + * duration for soft rt applications. + * + * In particular, doing this recharge now, i.e., + * before the weight-raising period for the + * application finishes, reduces the probability + * of the following negative scenario: + * 1) the weight of a soft rt application is + * raised at startup (as for any newly + * created application), + * 2) since the application is not interactive, + * at a certain time weight-raising is + * stopped for the application, + * 3) at that time the application happens to + * still have pending requests, and hence + * is destined to not have a chance to be + * deemed soft rt before these requests are + * completed (see the comments to the + * function bfq_bfqq_softrt_next_start() + * for details on soft rt detection), + * 4) these pending requests experience a high + * latency because the application is not + * weight-raised while they are pending. + */ + if (bfqq->wr_cur_max_time != + bfqd->bfq_wr_rt_max_time) { + bfqq->wr_start_at_switch_to_srt = + bfqq->last_wr_start_finish; + BUG_ON(time_is_after_jiffies(bfqq->last_wr_start_finish)); + + bfqq->wr_cur_max_time = + bfqd->bfq_wr_rt_max_time; + bfqq->wr_coeff = bfqd->bfq_wr_coeff * + BFQ_SOFTRT_WEIGHT_FACTOR; + bfq_log_bfqq(bfqd, bfqq, + "switching to soft_rt wr"); + } else + bfq_log_bfqq(bfqd, bfqq, + "moving forward soft_rt wr duration"); + bfqq->last_wr_start_finish = jiffies; + } + } +} + +static bool bfq_bfqq_idle_for_long_time(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + return bfqq->dispatched == 0 && + time_is_before_jiffies( + bfqq->budget_timeout + + bfqd->bfq_wr_min_idle_time); +} + +static void bfq_bfqq_handle_idle_busy_switch(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + int old_wr_coeff, + struct request *rq, + bool *interactive) +{ + bool soft_rt, in_burst, wr_or_deserves_wr, + bfqq_wants_to_preempt, + idle_for_long_time = bfq_bfqq_idle_for_long_time(bfqd, bfqq), + /* + * See the comments on + * bfq_bfqq_update_budg_for_activation for + * details on the usage of the next variable. + */ + arrived_in_time = ktime_get_ns() <= + RQ_BIC(rq)->ttime.last_end_request + + bfqd->bfq_slice_idle * 3; + + bfq_log_bfqq(bfqd, bfqq, + "bfq_add_request non-busy: " + "jiffies %lu, in_time %d, idle_long %d busyw %d " + "wr_coeff %u", + jiffies, arrived_in_time, + idle_for_long_time, + bfq_bfqq_non_blocking_wait_rq(bfqq), + old_wr_coeff); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + BUG_ON(bfqq == bfqd->in_service_queue); + bfqg_stats_update_io_add(bfqq_group(RQ_BFQQ(rq)), bfqq, rq->cmd_flags); + + /* + * bfqq deserves to be weight-raised if: + * - it is sync, + * - it does not belong to a large burst, + * - it has been idle for enough time or is soft real-time, + * - is linked to a bfq_io_cq (it is not shared in any sense) + */ + in_burst = bfq_bfqq_in_large_burst(bfqq); + soft_rt = bfqd->bfq_wr_max_softrt_rate > 0 && + !in_burst && + time_is_before_jiffies(bfqq->soft_rt_next_start); + *interactive = + !in_burst && + idle_for_long_time; + wr_or_deserves_wr = bfqd->low_latency && + (bfqq->wr_coeff > 1 || + (bfq_bfqq_sync(bfqq) && + bfqq->bic && (*interactive || soft_rt))); + + bfq_log_bfqq(bfqd, bfqq, + "bfq_add_request: " + "in_burst %d, " + "soft_rt %d (next %lu), inter %d, bic %p", + bfq_bfqq_in_large_burst(bfqq), soft_rt, + bfqq->soft_rt_next_start, + *interactive, + bfqq->bic); + + /* + * Using the last flag, update budget and check whether bfqq + * may want to preempt the in-service queue. + */ + bfqq_wants_to_preempt = + bfq_bfqq_update_budg_for_activation(bfqd, bfqq, + arrived_in_time, + wr_or_deserves_wr); + + /* + * If bfqq happened to be activated in a burst, but has been + * idle for much more than an interactive queue, then we + * assume that, in the overall I/O initiated in the burst, the + * I/O associated with bfqq is finished. So bfqq does not need + * to be treated as a queue belonging to a burst + * anymore. Accordingly, we reset bfqq's in_large_burst flag + * if set, and remove bfqq from the burst list if it's + * there. We do not decrement burst_size, because the fact + * that bfqq does not need to belong to the burst list any + * more does not invalidate the fact that bfqq was created in + * a burst. + */ + if (likely(!bfq_bfqq_just_created(bfqq)) && + idle_for_long_time && + time_is_before_jiffies( + bfqq->budget_timeout + + msecs_to_jiffies(10000))) { + hlist_del_init(&bfqq->burst_list_node); + bfq_clear_bfqq_in_large_burst(bfqq); + } + + bfq_clear_bfqq_just_created(bfqq); + + if (!bfq_bfqq_IO_bound(bfqq)) { + if (arrived_in_time) { + bfqq->requests_within_timer++; + if (bfqq->requests_within_timer >= + bfqd->bfq_requests_within_timer) + bfq_mark_bfqq_IO_bound(bfqq); + } else + bfqq->requests_within_timer = 0; + bfq_log_bfqq(bfqd, bfqq, "requests in time %d", + bfqq->requests_within_timer); + } + + if (bfqd->low_latency) { + if (unlikely(time_is_after_jiffies(bfqq->split_time))) + /* wraparound */ + bfqq->split_time = + jiffies - bfqd->bfq_wr_min_idle_time - 1; + + if (time_is_before_jiffies(bfqq->split_time + + bfqd->bfq_wr_min_idle_time)) { + bfq_update_bfqq_wr_on_rq_arrival(bfqd, bfqq, + old_wr_coeff, + wr_or_deserves_wr, + *interactive, + in_burst, + soft_rt); + + if (old_wr_coeff != bfqq->wr_coeff) + bfqq->entity.prio_changed = 1; + } + } + + bfqq->last_idle_bklogged = jiffies; + bfqq->service_from_backlogged = 0; + bfq_clear_bfqq_softrt_update(bfqq); + + bfq_add_bfqq_busy(bfqd, bfqq); + + /* + * Expire in-service queue only if preemption may be needed + * for guarantees. In this respect, the function + * next_queue_may_preempt just checks a simple, necessary + * condition, and not a sufficient condition based on + * timestamps. In fact, for the latter condition to be + * evaluated, timestamps would need first to be updated, and + * this operation is quite costly (see the comments on the + * function bfq_bfqq_update_budg_for_activation). + */ + if (bfqd->in_service_queue && bfqq_wants_to_preempt && + bfqd->in_service_queue->wr_coeff < bfqq->wr_coeff && + next_queue_may_preempt(bfqd)) { + struct bfq_queue *in_serv = + bfqd->in_service_queue; + BUG_ON(in_serv == bfqq); + + bfq_bfqq_expire(bfqd, bfqd->in_service_queue, + false, BFQ_BFQQ_PREEMPTED); + } +} + +static void bfq_add_request(struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + struct bfq_data *bfqd = bfqq->bfqd; + struct request *next_rq, *prev; + unsigned int old_wr_coeff = bfqq->wr_coeff; + bool interactive = false; + + bfq_log_bfqq(bfqd, bfqq, "add_request: size %u %s", + blk_rq_sectors(rq), rq_is_sync(rq) ? "S" : "A"); + + if (bfqq->wr_coeff > 1) /* queue is being weight-raised */ + bfq_log_bfqq(bfqd, bfqq, + "raising period dur %u/%u msec, old coeff %u, w %d(%d)", + jiffies_to_msecs(jiffies - bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time), + bfqq->wr_coeff, + bfqq->entity.weight, bfqq->entity.orig_weight); + + bfqq->queued[rq_is_sync(rq)]++; + bfqd->queued++; + + elv_rb_add(&bfqq->sort_list, rq); + + /* + * Check if this request is a better next-to-serve candidate. + */ + prev = bfqq->next_rq; + next_rq = bfq_choose_req(bfqd, bfqq->next_rq, rq, bfqd->last_position); + BUG_ON(!next_rq); + bfqq->next_rq = next_rq; + + /* + * Adjust priority tree position, if next_rq changes. + */ + if (prev != bfqq->next_rq) + bfq_pos_tree_add_move(bfqd, bfqq); + + if (!bfq_bfqq_busy(bfqq)) /* switching to busy ... */ + bfq_bfqq_handle_idle_busy_switch(bfqd, bfqq, old_wr_coeff, + rq, &interactive); + else { + if (bfqd->low_latency && old_wr_coeff == 1 && !rq_is_sync(rq) && + time_is_before_jiffies( + bfqq->last_wr_start_finish + + bfqd->bfq_wr_min_inter_arr_async)) { + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqd, bfqq, + "non-idle wrais starting, " + "wr_max_time %u wr_busy %d", + jiffies_to_msecs(bfqq->wr_cur_max_time), + bfqd->wr_busy_queues); + } + if (prev != bfqq->next_rq) + bfq_updated_next_req(bfqd, bfqq); + } + + /* + * Assign jiffies to last_wr_start_finish in the following + * cases: + * + * . if bfqq is not going to be weight-raised, because, for + * non weight-raised queues, last_wr_start_finish stores the + * arrival time of the last request; as of now, this piece + * of information is used only for deciding whether to + * weight-raise async queues + * + * . if bfqq is not weight-raised, because, if bfqq is now + * switching to weight-raised, then last_wr_start_finish + * stores the time when weight-raising starts + * + * . if bfqq is interactive, because, regardless of whether + * bfqq is currently weight-raised, the weight-raising + * period must start or restart (this case is considered + * separately because it is not detected by the above + * conditions, if bfqq is already weight-raised) + * + * last_wr_start_finish has to be updated also if bfqq is soft + * real-time, because the weight-raising period is constantly + * restarted on idle-to-busy transitions for these queues, but + * this is already done in bfq_bfqq_handle_idle_busy_switch if + * needed. + */ + if (bfqd->low_latency && + (old_wr_coeff == 1 || bfqq->wr_coeff == 1 || interactive)) + bfqq->last_wr_start_finish = jiffies; +} + +static struct request *bfq_find_rq_fmerge(struct bfq_data *bfqd, + struct bio *bio) +{ + struct task_struct *tsk = current; + struct bfq_io_cq *bic; + struct bfq_queue *bfqq; + + bic = bfq_bic_lookup(bfqd, tsk->io_context); + if (!bic) + return NULL; + + bfqq = bic_to_bfqq(bic, op_is_sync(bio->bi_opf)); + if (bfqq) + return elv_rb_find(&bfqq->sort_list, bio_end_sector(bio)); + + return NULL; +} + +static sector_t get_sdist(sector_t last_pos, struct request *rq) +{ + sector_t sdist = 0; + + if (last_pos) { + if (last_pos < blk_rq_pos(rq)) + sdist = blk_rq_pos(rq) - last_pos; + else + sdist = last_pos - blk_rq_pos(rq); + } + + return sdist; +} + +static void bfq_activate_request(struct request_queue *q, struct request *rq) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + bfqd->rq_in_driver++; +} + +static void bfq_deactivate_request(struct request_queue *q, struct request *rq) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + + BUG_ON(bfqd->rq_in_driver == 0); + bfqd->rq_in_driver--; +} + +static void bfq_remove_request(struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + struct bfq_data *bfqd = bfqq->bfqd; + const int sync = rq_is_sync(rq); + + BUG_ON(bfqq->entity.service > bfqq->entity.budget && + bfqq == bfqd->in_service_queue); + + if (bfqq->next_rq == rq) { + bfqq->next_rq = bfq_find_next_rq(bfqd, bfqq, rq); + bfq_updated_next_req(bfqd, bfqq); + } + + if (rq->queuelist.prev != &rq->queuelist) + list_del_init(&rq->queuelist); + BUG_ON(bfqq->queued[sync] == 0); + bfqq->queued[sync]--; + bfqd->queued--; + elv_rb_del(&bfqq->sort_list, rq); + + if (RB_EMPTY_ROOT(&bfqq->sort_list)) { + bfqq->next_rq = NULL; + + BUG_ON(bfqq->entity.budget < 0); + + if (bfq_bfqq_busy(bfqq) && bfqq != bfqd->in_service_queue) { + BUG_ON(bfqq->ref < 2); /* referred by rq and on tree */ + bfq_del_bfqq_busy(bfqd, bfqq, false); + /* + * bfqq emptied. In normal operation, when + * bfqq is empty, bfqq->entity.service and + * bfqq->entity.budget must contain, + * respectively, the service received and the + * budget used last time bfqq emptied. These + * facts do not hold in this case, as at least + * this last removal occurred while bfqq is + * not in service. To avoid inconsistencies, + * reset both bfqq->entity.service and + * bfqq->entity.budget, if bfqq has still a + * process that may issue I/O requests to it. + */ + bfqq->entity.budget = bfqq->entity.service = 0; + } + + /* + * Remove queue from request-position tree as it is empty. + */ + if (bfqq->pos_root) { + rb_erase(&bfqq->pos_node, bfqq->pos_root); + bfqq->pos_root = NULL; + } + } + + if (rq->cmd_flags & REQ_META) { + BUG_ON(bfqq->meta_pending == 0); + bfqq->meta_pending--; + } + bfqg_stats_update_io_remove(bfqq_group(bfqq), rq->cmd_flags); +} + +static enum elv_merge bfq_merge(struct request_queue *q, struct request **req, + struct bio *bio) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + struct request *__rq; + + __rq = bfq_find_rq_fmerge(bfqd, bio); + if (__rq && elv_bio_merge_ok(__rq, bio)) { + *req = __rq; + return ELEVATOR_FRONT_MERGE; + } + + return ELEVATOR_NO_MERGE; +} + +static void bfq_merged_request(struct request_queue *q, struct request *req, + enum elv_merge type) +{ + if (type == ELEVATOR_FRONT_MERGE && + rb_prev(&req->rb_node) && + blk_rq_pos(req) < + blk_rq_pos(container_of(rb_prev(&req->rb_node), + struct request, rb_node))) { + struct bfq_queue *bfqq = RQ_BFQQ(req); + struct bfq_data *bfqd = bfqq->bfqd; + struct request *prev, *next_rq; + + /* Reposition request in its sort_list */ + elv_rb_del(&bfqq->sort_list, req); + elv_rb_add(&bfqq->sort_list, req); + /* Choose next request to be served for bfqq */ + prev = bfqq->next_rq; + next_rq = bfq_choose_req(bfqd, bfqq->next_rq, req, + bfqd->last_position); + BUG_ON(!next_rq); + bfqq->next_rq = next_rq; + /* + * If next_rq changes, update both the queue's budget to + * fit the new request and the queue's position in its + * rq_pos_tree. + */ + if (prev != bfqq->next_rq) { + bfq_updated_next_req(bfqd, bfqq); + bfq_pos_tree_add_move(bfqd, bfqq); + } + } +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static void bfq_bio_merged(struct request_queue *q, struct request *req, + struct bio *bio) +{ + bfqg_stats_update_io_merged(bfqq_group(RQ_BFQQ(req)), bio->bi_opf); +} +#endif + +static void bfq_merged_requests(struct request_queue *q, struct request *rq, + struct request *next) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq), *next_bfqq = RQ_BFQQ(next); + + /* + * If next and rq belong to the same bfq_queue and next is older + * than rq, then reposition rq in the fifo (by substituting next + * with rq). Otherwise, if next and rq belong to different + * bfq_queues, never reposition rq: in fact, we would have to + * reposition it with respect to next's position in its own fifo, + * which would most certainly be too expensive with respect to + * the benefits. + */ + if (bfqq == next_bfqq && + !list_empty(&rq->queuelist) && !list_empty(&next->queuelist) && + next->fifo_time < rq->fifo_time) { + list_del_init(&rq->queuelist); + list_replace_init(&next->queuelist, &rq->queuelist); + rq->fifo_time = next->fifo_time; + } + + if (bfqq->next_rq == next) + bfqq->next_rq = rq; + + bfq_remove_request(next); + bfqg_stats_update_io_merged(bfqq_group(bfqq), next->cmd_flags); +} + +/* Must be called with bfqq != NULL */ +static void bfq_bfqq_end_wr(struct bfq_queue *bfqq) +{ + BUG_ON(!bfqq); + + if (bfq_bfqq_busy(bfqq)) { + bfqq->bfqd->wr_busy_queues--; + BUG_ON(bfqq->bfqd->wr_busy_queues < 0); + } + bfqq->wr_coeff = 1; + bfqq->wr_cur_max_time = 0; + bfqq->last_wr_start_finish = jiffies; + /* + * Trigger a weight change on the next invocation of + * __bfq_entity_update_weight_prio. + */ + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqq->bfqd, bfqq, + "end_wr: wrais ending at %lu, rais_max_time %u", + bfqq->last_wr_start_finish, + jiffies_to_msecs(bfqq->wr_cur_max_time)); + bfq_log_bfqq(bfqq->bfqd, bfqq, "end_wr: wr_busy %d", + bfqq->bfqd->wr_busy_queues); +} + +static void bfq_end_wr_async_queues(struct bfq_data *bfqd, + struct bfq_group *bfqg) +{ + int i, j; + + for (i = 0; i < 2; i++) + for (j = 0; j < IOPRIO_BE_NR; j++) + if (bfqg->async_bfqq[i][j]) + bfq_bfqq_end_wr(bfqg->async_bfqq[i][j]); + if (bfqg->async_idle_bfqq) + bfq_bfqq_end_wr(bfqg->async_idle_bfqq); +} + +static void bfq_end_wr(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq; + + spin_lock_irq(bfqd->queue->queue_lock); + + list_for_each_entry(bfqq, &bfqd->active_list, bfqq_list) + bfq_bfqq_end_wr(bfqq); + list_for_each_entry(bfqq, &bfqd->idle_list, bfqq_list) + bfq_bfqq_end_wr(bfqq); + bfq_end_wr_async(bfqd); + + spin_unlock_irq(bfqd->queue->queue_lock); +} + +static sector_t bfq_io_struct_pos(void *io_struct, bool request) +{ + if (request) + return blk_rq_pos(io_struct); + else + return ((struct bio *)io_struct)->bi_iter.bi_sector; +} + +static int bfq_rq_close_to_sector(void *io_struct, bool request, + sector_t sector) +{ + return abs(bfq_io_struct_pos(io_struct, request) - sector) <= + BFQQ_CLOSE_THR; +} + +static struct bfq_queue *bfqq_find_close(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + sector_t sector) +{ + struct rb_root *root = &bfq_bfqq_to_bfqg(bfqq)->rq_pos_tree; + struct rb_node *parent, *node; + struct bfq_queue *__bfqq; + + if (RB_EMPTY_ROOT(root)) + return NULL; + + /* + * First, if we find a request starting at the end of the last + * request, choose it. + */ + __bfqq = bfq_rq_pos_tree_lookup(bfqd, root, sector, &parent, NULL); + if (__bfqq) + return __bfqq; + + /* + * If the exact sector wasn't found, the parent of the NULL leaf + * will contain the closest sector (rq_pos_tree sorted by + * next_request position). + */ + __bfqq = rb_entry(parent, struct bfq_queue, pos_node); + if (bfq_rq_close_to_sector(__bfqq->next_rq, true, sector)) + return __bfqq; + + if (blk_rq_pos(__bfqq->next_rq) < sector) + node = rb_next(&__bfqq->pos_node); + else + node = rb_prev(&__bfqq->pos_node); + if (!node) + return NULL; + + __bfqq = rb_entry(node, struct bfq_queue, pos_node); + if (bfq_rq_close_to_sector(__bfqq->next_rq, true, sector)) + return __bfqq; + + return NULL; +} + +static struct bfq_queue *bfq_find_close_cooperator(struct bfq_data *bfqd, + struct bfq_queue *cur_bfqq, + sector_t sector) +{ + struct bfq_queue *bfqq; + + /* + * We shall notice if some of the queues are cooperating, + * e.g., working closely on the same area of the device. In + * that case, we can group them together and: 1) don't waste + * time idling, and 2) serve the union of their requests in + * the best possible order for throughput. + */ + bfqq = bfqq_find_close(bfqd, cur_bfqq, sector); + if (!bfqq || bfqq == cur_bfqq) + return NULL; + + return bfqq; +} + +static struct bfq_queue * +bfq_setup_merge(struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) +{ + int process_refs, new_process_refs; + struct bfq_queue *__bfqq; + + /* + * If there are no process references on the new_bfqq, then it is + * unsafe to follow the ->new_bfqq chain as other bfqq's in the chain + * may have dropped their last reference (not just their last process + * reference). + */ + if (!bfqq_process_refs(new_bfqq)) + return NULL; + + /* Avoid a circular list and skip interim queue merges. */ + while ((__bfqq = new_bfqq->new_bfqq)) { + if (__bfqq == bfqq) + return NULL; + new_bfqq = __bfqq; + } + + process_refs = bfqq_process_refs(bfqq); + new_process_refs = bfqq_process_refs(new_bfqq); + /* + * If the process for the bfqq has gone away, there is no + * sense in merging the queues. + */ + if (process_refs == 0 || new_process_refs == 0) + return NULL; + + bfq_log_bfqq(bfqq->bfqd, bfqq, "scheduling merge with queue %d", + new_bfqq->pid); + + /* + * Merging is just a redirection: the requests of the process + * owning one of the two queues are redirected to the other queue. + * The latter queue, in its turn, is set as shared if this is the + * first time that the requests of some process are redirected to + * it. + * + * We redirect bfqq to new_bfqq and not the opposite, because we + * are in the context of the process owning bfqq, hence we have + * the io_cq of this process. So we can immediately configure this + * io_cq to redirect the requests of the process to new_bfqq. + * + * NOTE, even if new_bfqq coincides with the in-service queue, the + * io_cq of new_bfqq is not available, because, if the in-service + * queue is shared, bfqd->in_service_bic may not point to the + * io_cq of the in-service queue. + * Redirecting the requests of the process owning bfqq to the + * currently in-service queue is in any case the best option, as + * we feed the in-service queue with new requests close to the + * last request served and, by doing so, hopefully increase the + * throughput. + */ + bfqq->new_bfqq = new_bfqq; + new_bfqq->ref += process_refs; + return new_bfqq; +} + +static bool bfq_may_be_close_cooperator(struct bfq_queue *bfqq, + struct bfq_queue *new_bfqq) +{ + if (bfq_class_idle(bfqq) || bfq_class_idle(new_bfqq) || + (bfqq->ioprio_class != new_bfqq->ioprio_class)) + return false; + + /* + * If either of the queues has already been detected as seeky, + * then merging it with the other queue is unlikely to lead to + * sequential I/O. + */ + if (BFQQ_SEEKY(bfqq) || BFQQ_SEEKY(new_bfqq)) + return false; + + /* + * Interleaved I/O is known to be done by (some) applications + * only for reads, so it does not make sense to merge async + * queues. + */ + if (!bfq_bfqq_sync(bfqq) || !bfq_bfqq_sync(new_bfqq)) + return false; + + return true; +} + +/* + * If this function returns true, then bfqq cannot be merged. The idea + * is that true cooperation happens very early after processes start + * to do I/O. Usually, late cooperations are just accidental false + * positives. In case bfqq is weight-raised, such false positives + * would evidently degrade latency guarantees for bfqq. + */ +static bool wr_from_too_long(struct bfq_queue *bfqq) +{ + return bfqq->wr_coeff > 1 && + time_is_before_jiffies(bfqq->last_wr_start_finish + + msecs_to_jiffies(100)); +} + +/* + * Attempt to schedule a merge of bfqq with the currently in-service + * queue or with a close queue among the scheduled queues. Return + * NULL if no merge was scheduled, a pointer to the shared bfq_queue + * structure otherwise. + * + * The OOM queue is not allowed to participate to cooperation: in fact, since + * the requests temporarily redirected to the OOM queue could be redirected + * again to dedicated queues at any time, the state needed to correctly + * handle merging with the OOM queue would be quite complex and expensive + * to maintain. Besides, in such a critical condition as an out of memory, + * the benefits of queue merging may be little relevant, or even negligible. + * + * Weight-raised queues can be merged only if their weight-raising + * period has just started. In fact cooperating processes are usually + * started together. Thus, with this filter we avoid false positives + * that would jeopardize low-latency guarantees. + * + * WARNING: queue merging may impair fairness among non-weight raised + * queues, for at least two reasons: 1) the original weight of a + * merged queue may change during the merged state, 2) even being the + * weight the same, a merged queue may be bloated with many more + * requests than the ones produced by its originally-associated + * process. + */ +static struct bfq_queue * +bfq_setup_cooperator(struct bfq_data *bfqd, struct bfq_queue *bfqq, + void *io_struct, bool request) +{ + struct bfq_queue *in_service_bfqq, *new_bfqq; + + if (bfqq->new_bfqq) + return bfqq->new_bfqq; + + if (io_struct && wr_from_too_long(bfqq) && + likely(bfqq != &bfqd->oom_bfqq)) + bfq_log_bfqq(bfqd, bfqq, + "would have looked for coop, but bfq%d wr", + bfqq->pid); + + if (!io_struct || + wr_from_too_long(bfqq) || + unlikely(bfqq == &bfqd->oom_bfqq)) + return NULL; + + /* If there is only one backlogged queue, don't search. */ + if (bfqd->busy_queues == 1) + return NULL; + + in_service_bfqq = bfqd->in_service_queue; + + if (in_service_bfqq && in_service_bfqq != bfqq && + bfqd->in_service_bic && wr_from_too_long(in_service_bfqq) + && likely(in_service_bfqq == &bfqd->oom_bfqq)) + bfq_log_bfqq(bfqd, bfqq, + "would have tried merge with in-service-queue, but wr"); + + if (!in_service_bfqq || in_service_bfqq == bfqq || + !bfqd->in_service_bic || wr_from_too_long(in_service_bfqq) || + unlikely(in_service_bfqq == &bfqd->oom_bfqq)) + goto check_scheduled; + + if (bfq_rq_close_to_sector(io_struct, request, bfqd->last_position) && + bfqq->entity.parent == in_service_bfqq->entity.parent && + bfq_may_be_close_cooperator(bfqq, in_service_bfqq)) { + new_bfqq = bfq_setup_merge(bfqq, in_service_bfqq); + if (new_bfqq) + return new_bfqq; + } + /* + * Check whether there is a cooperator among currently scheduled + * queues. The only thing we need is that the bio/request is not + * NULL, as we need it to establish whether a cooperator exists. + */ +check_scheduled: + new_bfqq = bfq_find_close_cooperator(bfqd, bfqq, + bfq_io_struct_pos(io_struct, request)); + + BUG_ON(new_bfqq && bfqq->entity.parent != new_bfqq->entity.parent); + + if (new_bfqq && wr_from_too_long(new_bfqq) && + likely(new_bfqq != &bfqd->oom_bfqq) && + bfq_may_be_close_cooperator(bfqq, new_bfqq)) + bfq_log_bfqq(bfqd, bfqq, + "would have merged with bfq%d, but wr", + new_bfqq->pid); + + if (new_bfqq && !wr_from_too_long(new_bfqq) && + likely(new_bfqq != &bfqd->oom_bfqq) && + bfq_may_be_close_cooperator(bfqq, new_bfqq)) + return bfq_setup_merge(bfqq, new_bfqq); + + return NULL; +} + +static void bfq_bfqq_save_state(struct bfq_queue *bfqq) +{ + struct bfq_io_cq *bic = bfqq->bic; + + /* + * If !bfqq->bic, the queue is already shared or its requests + * have already been redirected to a shared queue; both idle window + * and weight raising state have already been saved. Do nothing. + */ + if (!bic) + return; + + bic->saved_has_short_ttime = bfq_bfqq_has_short_ttime(bfqq); + bic->saved_IO_bound = bfq_bfqq_IO_bound(bfqq); + bic->saved_in_large_burst = bfq_bfqq_in_large_burst(bfqq); + bic->was_in_burst_list = !hlist_unhashed(&bfqq->burst_list_node); + bic->saved_wr_coeff = bfqq->wr_coeff; + bic->saved_wr_start_at_switch_to_srt = bfqq->wr_start_at_switch_to_srt; + bic->saved_last_wr_start_finish = bfqq->last_wr_start_finish; + bic->saved_wr_cur_max_time = bfqq->wr_cur_max_time; + BUG_ON(time_is_after_jiffies(bfqq->last_wr_start_finish)); +} + +static void bfq_get_bic_reference(struct bfq_queue *bfqq) +{ + /* + * If bfqq->bic has a non-NULL value, the bic to which it belongs + * is about to begin using a shared bfq_queue. + */ + if (bfqq->bic) + atomic_long_inc(&bfqq->bic->icq.ioc->refcount); +} + +static void +bfq_merge_bfqqs(struct bfq_data *bfqd, struct bfq_io_cq *bic, + struct bfq_queue *bfqq, struct bfq_queue *new_bfqq) +{ + bfq_log_bfqq(bfqd, bfqq, "merging with queue %lu", + (unsigned long) new_bfqq->pid); + /* Save weight raising and idle window of the merged queues */ + bfq_bfqq_save_state(bfqq); + bfq_bfqq_save_state(new_bfqq); + if (bfq_bfqq_IO_bound(bfqq)) + bfq_mark_bfqq_IO_bound(new_bfqq); + bfq_clear_bfqq_IO_bound(bfqq); + + /* + * If bfqq is weight-raised, then let new_bfqq inherit + * weight-raising. To reduce false positives, neglect the case + * where bfqq has just been created, but has not yet made it + * to be weight-raised (which may happen because EQM may merge + * bfqq even before bfq_add_request is executed for the first + * time for bfqq). Handling this case would however be very + * easy, thanks to the flag just_created. + */ + if (new_bfqq->wr_coeff == 1 && bfqq->wr_coeff > 1) { + new_bfqq->wr_coeff = bfqq->wr_coeff; + new_bfqq->wr_cur_max_time = bfqq->wr_cur_max_time; + new_bfqq->last_wr_start_finish = bfqq->last_wr_start_finish; + new_bfqq->wr_start_at_switch_to_srt = + bfqq->wr_start_at_switch_to_srt; + if (bfq_bfqq_busy(new_bfqq)) { + bfqd->wr_busy_queues++; + BUG_ON(bfqd->wr_busy_queues > bfqd->busy_queues); + } + + new_bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqd, new_bfqq, + "wr start after merge with %d, rais_max_time %u", + bfqq->pid, + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } + + if (bfqq->wr_coeff > 1) { /* bfqq has given its wr to new_bfqq */ + bfqq->wr_coeff = 1; + bfqq->entity.prio_changed = 1; + if (bfq_bfqq_busy(bfqq)) { + bfqd->wr_busy_queues--; + BUG_ON(bfqd->wr_busy_queues < 0); + } + + } + + bfq_log_bfqq(bfqd, new_bfqq, "merge_bfqqs: wr_busy %d", + bfqd->wr_busy_queues); + + /* + * Grab a reference to the bic, to prevent it from being destroyed + * before being possibly touched by a bfq_split_bfqq(). + */ + bfq_get_bic_reference(bfqq); + bfq_get_bic_reference(new_bfqq); + /* + * Merge queues (that is, let bic redirect its requests to new_bfqq) + */ + bic_set_bfqq(bic, new_bfqq, 1); + bfq_mark_bfqq_coop(new_bfqq); + /* + * new_bfqq now belongs to at least two bics (it is a shared queue): + * set new_bfqq->bic to NULL. bfqq either: + * - does not belong to any bic any more, and hence bfqq->bic must + * be set to NULL, or + * - is a queue whose owning bics have already been redirected to a + * different queue, hence the queue is destined to not belong to + * any bic soon and bfqq->bic is already NULL (therefore the next + * assignment causes no harm). + */ + new_bfqq->bic = NULL; + bfqq->bic = NULL; + /* release process reference to bfqq */ + bfq_put_queue(bfqq); +} + +static int bfq_allow_bio_merge(struct request_queue *q, struct request *rq, + struct bio *bio) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + bool is_sync = op_is_sync(bio->bi_opf); + struct bfq_io_cq *bic; + struct bfq_queue *bfqq, *new_bfqq; + + /* + * Disallow merge of a sync bio into an async request. + */ + if (is_sync && !rq_is_sync(rq)) + return false; + + /* + * Lookup the bfqq that this bio will be queued with. Allow + * merge only if rq is queued there. + * Queue lock is held here. + */ + bic = bfq_bic_lookup(bfqd, current->io_context); + if (!bic) + return false; + + bfqq = bic_to_bfqq(bic, is_sync); + /* + * We take advantage of this function to perform an early merge + * of the queues of possible cooperating processes. + */ + if (bfqq) { + new_bfqq = bfq_setup_cooperator(bfqd, bfqq, bio, false); + if (new_bfqq) { + bfq_merge_bfqqs(bfqd, bic, bfqq, new_bfqq); + /* + * If we get here, the bio will be queued in the + * shared queue, i.e., new_bfqq, so use new_bfqq + * to decide whether bio and rq can be merged. + */ + bfqq = new_bfqq; + } + } + + return bfqq == RQ_BFQQ(rq); +} + +static int bfq_allow_rq_merge(struct request_queue *q, struct request *rq, + struct request *next) +{ + return RQ_BFQQ(rq) == RQ_BFQQ(next); +} + +/* + * Set the maximum time for the in-service queue to consume its + * budget. This prevents seeky processes from lowering the throughput. + * In practice, a time-slice service scheme is used with seeky + * processes. + */ +static void bfq_set_budget_timeout(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + unsigned int timeout_coeff; + + if (bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time) + timeout_coeff = 1; + else + timeout_coeff = bfqq->entity.weight / bfqq->entity.orig_weight; + + bfqd->last_budget_start = ktime_get(); + + bfqq->budget_timeout = jiffies + + bfqd->bfq_timeout * timeout_coeff; + + bfq_log_bfqq(bfqd, bfqq, "set budget_timeout %u", + jiffies_to_msecs(bfqd->bfq_timeout * timeout_coeff)); +} + +static void __bfq_set_in_service_queue(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + if (bfqq) { + bfqg_stats_update_avg_queue_size(bfqq_group(bfqq)); + bfq_mark_bfqq_must_alloc(bfqq); + bfq_clear_bfqq_fifo_expire(bfqq); + + bfqd->budgets_assigned = (bfqd->budgets_assigned*7 + 256) / 8; + + BUG_ON(bfqq == bfqd->in_service_queue); + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + + if (time_is_before_jiffies(bfqq->last_wr_start_finish) && + bfqq->wr_coeff > 1 && + bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time && + time_is_before_jiffies(bfqq->budget_timeout)) { + /* + * For soft real-time queues, move the start + * of the weight-raising period forward by the + * time the queue has not received any + * service. Otherwise, a relatively long + * service delay is likely to cause the + * weight-raising period of the queue to end, + * because of the short duration of the + * weight-raising period of a soft real-time + * queue. It is worth noting that this move + * is not so dangerous for the other queues, + * because soft real-time queues are not + * greedy. + * + * To not add a further variable, we use the + * overloaded field budget_timeout to + * determine for how long the queue has not + * received service, i.e., how much time has + * elapsed since the queue expired. However, + * this is a little imprecise, because + * budget_timeout is set to jiffies if bfqq + * not only expires, but also remains with no + * request. + */ + if (time_after(bfqq->budget_timeout, + bfqq->last_wr_start_finish)) + bfqq->last_wr_start_finish += + jiffies - bfqq->budget_timeout; + else + bfqq->last_wr_start_finish = jiffies; + + if (time_is_after_jiffies(bfqq->last_wr_start_finish)) { + pr_crit( + "BFQ WARNING:last %lu budget %lu jiffies %lu", + bfqq->last_wr_start_finish, + bfqq->budget_timeout, + jiffies); + pr_crit("diff %lu", jiffies - + max_t(unsigned long, + bfqq->last_wr_start_finish, + bfqq->budget_timeout)); + bfqq->last_wr_start_finish = jiffies; + } + } + + bfq_set_budget_timeout(bfqd, bfqq); + bfq_log_bfqq(bfqd, bfqq, + "set_in_service_queue, cur-budget = %d", + bfqq->entity.budget); + } else + bfq_log(bfqd, "set_in_service_queue: NULL"); + + bfqd->in_service_queue = bfqq; +} + +/* + * Get and set a new queue for service. + */ +static struct bfq_queue *bfq_set_in_service_queue(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq = bfq_get_next_queue(bfqd); + + __bfq_set_in_service_queue(bfqd, bfqq); + return bfqq; +} + +static void bfq_arm_slice_timer(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq = bfqd->in_service_queue; + struct bfq_io_cq *bic; + u32 sl; + + BUG_ON(!RB_EMPTY_ROOT(&bfqq->sort_list)); + + /* Processes have exited, don't wait. */ + bic = bfqd->in_service_bic; + if (!bic || atomic_read(&bic->icq.ioc->active_ref) == 0) + return; + + bfq_mark_bfqq_wait_request(bfqq); + + /* + * We don't want to idle for seeks, but we do want to allow + * fair distribution of slice time for a process doing back-to-back + * seeks. So allow a little bit of time for him to submit a new rq. + * + * To prevent processes with (partly) seeky workloads from + * being too ill-treated, grant them a small fraction of the + * assigned budget before reducing the waiting time to + * BFQ_MIN_TT. This happened to help reduce latency. + */ + sl = bfqd->bfq_slice_idle; + /* + * Unless the queue is being weight-raised or the scenario is + * asymmetric, grant only minimum idle time if the queue + * is seeky. A long idling is preserved for a weight-raised + * queue, or, more in general, in an asymemtric scenario, + * because a long idling is needed for guaranteeing to a queue + * its reserved share of the throughput (in particular, it is + * needed if the queue has a higher weight than some other + * queue). + */ + if (BFQQ_SEEKY(bfqq) && bfqq->wr_coeff == 1 && + bfq_symmetric_scenario(bfqd)) + sl = min_t(u32, sl, BFQ_MIN_TT); + + bfqd->last_idling_start = ktime_get(); + hrtimer_start(&bfqd->idle_slice_timer, ns_to_ktime(sl), + HRTIMER_MODE_REL); + bfqg_stats_set_start_idle_time(bfqq_group(bfqq)); + bfq_log(bfqd, "arm idle: %ld/%ld ms", + sl / NSEC_PER_MSEC, bfqd->bfq_slice_idle / NSEC_PER_MSEC); +} + +/* + * In autotuning mode, max_budget is dynamically recomputed as the + * amount of sectors transferred in timeout at the estimated peak + * rate. This enables BFQ to utilize a full timeslice with a full + * budget, even if the in-service queue is served at peak rate. And + * this maximises throughput with sequential workloads. + */ +static unsigned long bfq_calc_max_budget(struct bfq_data *bfqd) +{ + return (u64)bfqd->peak_rate * USEC_PER_MSEC * + jiffies_to_msecs(bfqd->bfq_timeout)>>BFQ_RATE_SHIFT; +} + +/* + * Update parameters related to throughput and responsiveness, as a + * function of the estimated peak rate. See comments on + * bfq_calc_max_budget(), and on T_slow and T_fast arrays. + */ +static void update_thr_responsiveness_params(struct bfq_data *bfqd) +{ + int dev_type = blk_queue_nonrot(bfqd->queue); + + if (bfqd->bfq_user_max_budget == 0) { + bfqd->bfq_max_budget = + bfq_calc_max_budget(bfqd); + BUG_ON(bfqd->bfq_max_budget < 0); + bfq_log(bfqd, "new max_budget = %d", + bfqd->bfq_max_budget); + } + + if (bfqd->device_speed == BFQ_BFQD_FAST && + bfqd->peak_rate < device_speed_thresh[dev_type]) { + bfqd->device_speed = BFQ_BFQD_SLOW; + bfqd->RT_prod = R_slow[dev_type] * + T_slow[dev_type]; + } else if (bfqd->device_speed == BFQ_BFQD_SLOW && + bfqd->peak_rate > device_speed_thresh[dev_type]) { + bfqd->device_speed = BFQ_BFQD_FAST; + bfqd->RT_prod = R_fast[dev_type] * + T_fast[dev_type]; + } + + bfq_log(bfqd, +"dev_type %s dev_speed_class = %s (%llu sects/sec), thresh %llu setcs/sec", + dev_type == 0 ? "ROT" : "NONROT", + bfqd->device_speed == BFQ_BFQD_FAST ? "FAST" : "SLOW", + bfqd->device_speed == BFQ_BFQD_FAST ? + (USEC_PER_SEC*(u64)R_fast[dev_type])>>BFQ_RATE_SHIFT : + (USEC_PER_SEC*(u64)R_slow[dev_type])>>BFQ_RATE_SHIFT, + (USEC_PER_SEC*(u64)device_speed_thresh[dev_type])>> + BFQ_RATE_SHIFT); +} + +static void bfq_reset_rate_computation(struct bfq_data *bfqd, struct request *rq) +{ + if (rq != NULL) { /* new rq dispatch now, reset accordingly */ + bfqd->last_dispatch = bfqd->first_dispatch = ktime_get_ns() ; + bfqd->peak_rate_samples = 1; + bfqd->sequential_samples = 0; + bfqd->tot_sectors_dispatched = bfqd->last_rq_max_size = + blk_rq_sectors(rq); + } else /* no new rq dispatched, just reset the number of samples */ + bfqd->peak_rate_samples = 0; /* full re-init on next disp. */ + + bfq_log(bfqd, + "reset_rate_computation at end, sample %u/%u tot_sects %llu", + bfqd->peak_rate_samples, bfqd->sequential_samples, + bfqd->tot_sectors_dispatched); +} + +static void bfq_update_rate_reset(struct bfq_data *bfqd, struct request *rq) +{ + u32 rate, weight, divisor; + + /* + * For the convergence property to hold (see comments on + * bfq_update_peak_rate()) and for the assessment to be + * reliable, a minimum number of samples must be present, and + * a minimum amount of time must have elapsed. If not so, do + * not compute new rate. Just reset parameters, to get ready + * for a new evaluation attempt. + */ + if (bfqd->peak_rate_samples < BFQ_RATE_MIN_SAMPLES || + bfqd->delta_from_first < BFQ_RATE_MIN_INTERVAL) { + bfq_log(bfqd, + "update_rate_reset: only resetting, delta_first %lluus samples %d", + bfqd->delta_from_first>>10, bfqd->peak_rate_samples); + goto reset_computation; + } + + /* + * If a new request completion has occurred after last + * dispatch, then, to approximate the rate at which requests + * have been served by the device, it is more precise to + * extend the observation interval to the last completion. + */ + bfqd->delta_from_first = + max_t(u64, bfqd->delta_from_first, + bfqd->last_completion - bfqd->first_dispatch); + + BUG_ON(bfqd->delta_from_first == 0); + /* + * Rate computed in sects/usec, and not sects/nsec, for + * precision issues. + */ + rate = div64_ul(bfqd->tot_sectors_dispatched<delta_from_first, NSEC_PER_USEC)); + + bfq_log(bfqd, +"update_rate_reset: tot_sects %llu delta_first %lluus rate %llu sects/s (%d)", + bfqd->tot_sectors_dispatched, bfqd->delta_from_first>>10, + ((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT), + rate > 20< 20M sectors/sec) + */ + if ((bfqd->sequential_samples < (3 * bfqd->peak_rate_samples)>>2 && + rate <= bfqd->peak_rate) || + rate > 20<peak_rate_samples, bfqd->sequential_samples, + ((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT), + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT)); + goto reset_computation; + } else { + bfq_log(bfqd, + "update_rate_reset: do update, samples %u/%u rate/peak %llu/%llu", + bfqd->peak_rate_samples, bfqd->sequential_samples, + ((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT), + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT)); + } + + /* + * We have to update the peak rate, at last! To this purpose, + * we use a low-pass filter. We compute the smoothing constant + * of the filter as a function of the 'weight' of the new + * measured rate. + * + * As can be seen in next formulas, we define this weight as a + * quantity proportional to how sequential the workload is, + * and to how long the observation time interval is. + * + * The weight runs from 0 to 8. The maximum value of the + * weight, 8, yields the minimum value for the smoothing + * constant. At this minimum value for the smoothing constant, + * the measured rate contributes for half of the next value of + * the estimated peak rate. + * + * So, the first step is to compute the weight as a function + * of how sequential the workload is. Note that the weight + * cannot reach 9, because bfqd->sequential_samples cannot + * become equal to bfqd->peak_rate_samples, which, in its + * turn, holds true because bfqd->sequential_samples is not + * incremented for the first sample. + */ + weight = (9 * bfqd->sequential_samples) / bfqd->peak_rate_samples; + + /* + * Second step: further refine the weight as a function of the + * duration of the observation interval. + */ + weight = min_t(u32, 8, + div_u64(weight * bfqd->delta_from_first, + BFQ_RATE_REF_INTERVAL)); + + /* + * Divisor ranging from 10, for minimum weight, to 2, for + * maximum weight. + */ + divisor = 10 - weight; + BUG_ON(divisor == 0); + + /* + * Finally, update peak rate: + * + * peak_rate = peak_rate * (divisor-1) / divisor + rate / divisor + */ + bfqd->peak_rate *= divisor-1; + bfqd->peak_rate /= divisor; + rate /= divisor; /* smoothing constant alpha = 1/divisor */ + + bfq_log(bfqd, + "update_rate_reset: divisor %d tmp_peak_rate %llu tmp_rate %u", + divisor, + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT), + (u32)((USEC_PER_SEC*(u64)rate)>>BFQ_RATE_SHIFT)); + + BUG_ON(bfqd->peak_rate == 0); + BUG_ON(bfqd->peak_rate > 20<peak_rate += rate; + update_thr_responsiveness_params(bfqd); + BUG_ON(bfqd->peak_rate > 20<peak_rate_samples == 0) { /* first dispatch */ + bfq_log(bfqd, + "update_peak_rate: goto reset, samples %d", + bfqd->peak_rate_samples) ; + bfq_reset_rate_computation(bfqd, rq); + goto update_last_values; /* will add one sample */ + } + + /* + * Device idle for very long: the observation interval lasting + * up to this dispatch cannot be a valid observation interval + * for computing a new peak rate (similarly to the late- + * completion event in bfq_completed_request()). Go to + * update_rate_and_reset to have the following three steps + * taken: + * - close the observation interval at the last (previous) + * request dispatch or completion + * - compute rate, if possible, for that observation interval + * - start a new observation interval with this dispatch + */ + if (now_ns - bfqd->last_dispatch > 100*NSEC_PER_MSEC && + bfqd->rq_in_driver == 0) { + bfq_log(bfqd, +"update_peak_rate: jumping to updating&resetting delta_last %lluus samples %d", + (now_ns - bfqd->last_dispatch)>>10, + bfqd->peak_rate_samples) ; + goto update_rate_and_reset; + } + + /* Update sampling information */ + bfqd->peak_rate_samples++; + + if ((bfqd->rq_in_driver > 0 || + now_ns - bfqd->last_completion < BFQ_MIN_TT) + && get_sdist(bfqd->last_position, rq) < BFQQ_SEEK_THR) + bfqd->sequential_samples++; + + bfqd->tot_sectors_dispatched += blk_rq_sectors(rq); + + /* Reset max observed rq size every 32 dispatches */ + if (likely(bfqd->peak_rate_samples % 32)) + bfqd->last_rq_max_size = + max_t(u32, blk_rq_sectors(rq), bfqd->last_rq_max_size); + else + bfqd->last_rq_max_size = blk_rq_sectors(rq); + + bfqd->delta_from_first = now_ns - bfqd->first_dispatch; + + bfq_log(bfqd, + "update_peak_rate: added samples %u/%u tot_sects %llu delta_first %lluus", + bfqd->peak_rate_samples, bfqd->sequential_samples, + bfqd->tot_sectors_dispatched, + bfqd->delta_from_first>>10); + + /* Target observation interval not yet reached, go on sampling */ + if (bfqd->delta_from_first < BFQ_RATE_REF_INTERVAL) + goto update_last_values; + +update_rate_and_reset: + bfq_update_rate_reset(bfqd, rq); +update_last_values: + bfqd->last_position = blk_rq_pos(rq) + blk_rq_sectors(rq); + bfqd->last_dispatch = now_ns; + + bfq_log(bfqd, + "update_peak_rate: delta_first %lluus last_pos %llu peak_rate %llu", + (now_ns - bfqd->first_dispatch)>>10, + (unsigned long long) bfqd->last_position, + ((USEC_PER_SEC*(u64)bfqd->peak_rate)>>BFQ_RATE_SHIFT)); + bfq_log(bfqd, + "update_peak_rate: samples at end %d", bfqd->peak_rate_samples); +} + +/* + * Move request from internal lists to the dispatch list of the request queue + */ +static void bfq_dispatch_insert(struct request_queue *q, struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + + /* + * For consistency, the next instruction should have been executed + * after removing the request from the queue and dispatching it. + * We execute instead this instruction before bfq_remove_request() + * (and hence introduce a temporary inconsistency), for efficiency. + * In fact, in a forced_dispatch, this prevents two counters related + * to bfqq->dispatched to risk to be uselessly decremented if bfqq + * is not in service, and then to be incremented again after + * incrementing bfqq->dispatched. + */ + bfqq->dispatched++; + bfq_update_peak_rate(q->elevator->elevator_data, rq); + + bfq_remove_request(rq); + elv_dispatch_sort(q, rq); +} + +static void __bfq_bfqq_expire(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + BUG_ON(bfqq != bfqd->in_service_queue); + + /* + * If this bfqq is shared between multiple processes, check + * to make sure that those processes are still issuing I/Os + * within the mean seek distance. If not, it may be time to + * break the queues apart again. + */ + if (bfq_bfqq_coop(bfqq) && BFQQ_SEEKY(bfqq)) + bfq_mark_bfqq_split_coop(bfqq); + + if (RB_EMPTY_ROOT(&bfqq->sort_list)) { + if (bfqq->dispatched == 0) + /* + * Overloading budget_timeout field to store + * the time at which the queue remains with no + * backlog and no outstanding request; used by + * the weight-raising mechanism. + */ + bfqq->budget_timeout = jiffies; + + bfq_del_bfqq_busy(bfqd, bfqq, true); + } else { + bfq_requeue_bfqq(bfqd, bfqq, true); + /* + * Resort priority tree of potential close cooperators. + */ + bfq_pos_tree_add_move(bfqd, bfqq); + } + + /* + * All in-service entities must have been properly deactivated + * or requeued before executing the next function, which + * resets all in-service entites as no more in service. + */ + __bfq_bfqd_reset_in_service(bfqd); +} + +/** + * __bfq_bfqq_recalc_budget - try to adapt the budget to the @bfqq behavior. + * @bfqd: device data. + * @bfqq: queue to update. + * @reason: reason for expiration. + * + * Handle the feedback on @bfqq budget at queue expiration. + * See the body for detailed comments. + */ +static void __bfq_bfqq_recalc_budget(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + enum bfqq_expiration reason) +{ + struct request *next_rq; + int budget, min_budget; + + BUG_ON(bfqq != bfqd->in_service_queue); + + min_budget = bfq_min_budget(bfqd); + + if (bfqq->wr_coeff == 1) + budget = bfqq->max_budget; + else /* + * Use a constant, low budget for weight-raised queues, + * to help achieve a low latency. Keep it slightly higher + * than the minimum possible budget, to cause a little + * bit fewer expirations. + */ + budget = 2 * min_budget; + + bfq_log_bfqq(bfqd, bfqq, "recalc_budg: last budg %d, budg left %d", + bfqq->entity.budget, bfq_bfqq_budget_left(bfqq)); + bfq_log_bfqq(bfqd, bfqq, "recalc_budg: last max_budg %d, min budg %d", + budget, bfq_min_budget(bfqd)); + bfq_log_bfqq(bfqd, bfqq, "recalc_budg: sync %d, seeky %d", + bfq_bfqq_sync(bfqq), BFQQ_SEEKY(bfqd->in_service_queue)); + + if (bfq_bfqq_sync(bfqq) && bfqq->wr_coeff == 1) { + switch (reason) { + /* + * Caveat: in all the following cases we trade latency + * for throughput. + */ + case BFQ_BFQQ_TOO_IDLE: + /* + * This is the only case where we may reduce + * the budget: if there is no request of the + * process still waiting for completion, then + * we assume (tentatively) that the timer has + * expired because the batch of requests of + * the process could have been served with a + * smaller budget. Hence, betting that + * process will behave in the same way when it + * becomes backlogged again, we reduce its + * next budget. As long as we guess right, + * this budget cut reduces the latency + * experienced by the process. + * + * However, if there are still outstanding + * requests, then the process may have not yet + * issued its next request just because it is + * still waiting for the completion of some of + * the still outstanding ones. So in this + * subcase we do not reduce its budget, on the + * contrary we increase it to possibly boost + * the throughput, as discussed in the + * comments to the BUDGET_TIMEOUT case. + */ + if (bfqq->dispatched > 0) /* still outstanding reqs */ + budget = min(budget * 2, bfqd->bfq_max_budget); + else { + if (budget > 5 * min_budget) + budget -= 4 * min_budget; + else + budget = min_budget; + } + break; + case BFQ_BFQQ_BUDGET_TIMEOUT: + /* + * We double the budget here because it gives + * the chance to boost the throughput if this + * is not a seeky process (and has bumped into + * this timeout because of, e.g., ZBR). + */ + budget = min(budget * 2, bfqd->bfq_max_budget); + break; + case BFQ_BFQQ_BUDGET_EXHAUSTED: + /* + * The process still has backlog, and did not + * let either the budget timeout or the disk + * idling timeout expire. Hence it is not + * seeky, has a short thinktime and may be + * happy with a higher budget too. So + * definitely increase the budget of this good + * candidate to boost the disk throughput. + */ + budget = min(budget * 4, bfqd->bfq_max_budget); + break; + case BFQ_BFQQ_NO_MORE_REQUESTS: + /* + * For queues that expire for this reason, it + * is particularly important to keep the + * budget close to the actual service they + * need. Doing so reduces the timestamp + * misalignment problem described in the + * comments in the body of + * __bfq_activate_entity. In fact, suppose + * that a queue systematically expires for + * BFQ_BFQQ_NO_MORE_REQUESTS and presents a + * new request in time to enjoy timestamp + * back-shifting. The larger the budget of the + * queue is with respect to the service the + * queue actually requests in each service + * slot, the more times the queue can be + * reactivated with the same virtual finish + * time. It follows that, even if this finish + * time is pushed to the system virtual time + * to reduce the consequent timestamp + * misalignment, the queue unjustly enjoys for + * many re-activations a lower finish time + * than all newly activated queues. + * + * The service needed by bfqq is measured + * quite precisely by bfqq->entity.service. + * Since bfqq does not enjoy device idling, + * bfqq->entity.service is equal to the number + * of sectors that the process associated with + * bfqq requested to read/write before waiting + * for request completions, or blocking for + * other reasons. + */ + budget = max_t(int, bfqq->entity.service, min_budget); + break; + default: + return; + } + } else if (!bfq_bfqq_sync(bfqq)) + /* + * Async queues get always the maximum possible + * budget, as for them we do not care about latency + * (in addition, their ability to dispatch is limited + * by the charging factor). + */ + budget = bfqd->bfq_max_budget; + + bfqq->max_budget = budget; + + if (bfqd->budgets_assigned >= bfq_stats_min_budgets && + !bfqd->bfq_user_max_budget) + bfqq->max_budget = min(bfqq->max_budget, bfqd->bfq_max_budget); + + /* + * If there is still backlog, then assign a new budget, making + * sure that it is large enough for the next request. Since + * the finish time of bfqq must be kept in sync with the + * budget, be sure to call __bfq_bfqq_expire() *after* this + * update. + * + * If there is no backlog, then no need to update the budget; + * it will be updated on the arrival of a new request. + */ + next_rq = bfqq->next_rq; + if (next_rq) { + BUG_ON(reason == BFQ_BFQQ_TOO_IDLE || + reason == BFQ_BFQQ_NO_MORE_REQUESTS); + bfqq->entity.budget = max_t(unsigned long, bfqq->max_budget, + bfq_serv_to_charge(next_rq, bfqq)); + BUG_ON(!bfq_bfqq_busy(bfqq)); + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + } + + bfq_log_bfqq(bfqd, bfqq, "head sect: %u, new budget %d", + next_rq ? blk_rq_sectors(next_rq) : 0, + bfqq->entity.budget); +} + +/* + * Return true if the process associated with bfqq is "slow". The slow + * flag is used, in addition to the budget timeout, to reduce the + * amount of service provided to seeky processes, and thus reduce + * their chances to lower the throughput. More details in the comments + * on the function bfq_bfqq_expire(). + * + * An important observation is in order: as discussed in the comments + * on the function bfq_update_peak_rate(), with devices with internal + * queues, it is hard if ever possible to know when and for how long + * an I/O request is processed by the device (apart from the trivial + * I/O pattern where a new request is dispatched only after the + * previous one has been completed). This makes it hard to evaluate + * the real rate at which the I/O requests of each bfq_queue are + * served. In fact, for an I/O scheduler like BFQ, serving a + * bfq_queue means just dispatching its requests during its service + * slot (i.e., until the budget of the queue is exhausted, or the + * queue remains idle, or, finally, a timeout fires). But, during the + * service slot of a bfq_queue, around 100 ms at most, the device may + * be even still processing requests of bfq_queues served in previous + * service slots. On the opposite end, the requests of the in-service + * bfq_queue may be completed after the service slot of the queue + * finishes. + * + * Anyway, unless more sophisticated solutions are used + * (where possible), the sum of the sizes of the requests dispatched + * during the service slot of a bfq_queue is probably the only + * approximation available for the service received by the bfq_queue + * during its service slot. And this sum is the quantity used in this + * function to evaluate the I/O speed of a process. + */ +static bool bfq_bfqq_is_slow(struct bfq_data *bfqd, struct bfq_queue *bfqq, + bool compensate, enum bfqq_expiration reason, + unsigned long *delta_ms) +{ + ktime_t delta_ktime; + u32 delta_usecs; + bool slow = BFQQ_SEEKY(bfqq); /* if delta too short, use seekyness */ + + if (!bfq_bfqq_sync(bfqq)) + return false; + + if (compensate) + delta_ktime = bfqd->last_idling_start; + else + delta_ktime = ktime_get(); + delta_ktime = ktime_sub(delta_ktime, bfqd->last_budget_start); + delta_usecs = ktime_to_us(delta_ktime); + + /* don't use too short time intervals */ + if (delta_usecs < 1000) { + if (blk_queue_nonrot(bfqd->queue)) + /* + * give same worst-case guarantees as idling + * for seeky + */ + *delta_ms = BFQ_MIN_TT / NSEC_PER_MSEC; + else /* charge at least one seek */ + *delta_ms = bfq_slice_idle / NSEC_PER_MSEC; + + bfq_log(bfqd, "bfq_bfqq_is_slow: too short %u", delta_usecs); + + return slow; + } + + *delta_ms = delta_usecs / USEC_PER_MSEC; + + /* + * Use only long (> 20ms) intervals to filter out excessive + * spikes in service rate estimation. + */ + if (delta_usecs > 20000) { + /* + * Caveat for rotational devices: processes doing I/O + * in the slower disk zones tend to be slow(er) even + * if not seeky. In this respect, the estimated peak + * rate is likely to be an average over the disk + * surface. Accordingly, to not be too harsh with + * unlucky processes, a process is deemed slow only if + * its rate has been lower than half of the estimated + * peak rate. + */ + slow = bfqq->entity.service < bfqd->bfq_max_budget / 2; + bfq_log(bfqd, "bfq_bfqq_is_slow: relative rate %d/%d", + bfqq->entity.service, bfqd->bfq_max_budget); + } + + bfq_log_bfqq(bfqd, bfqq, "bfq_bfqq_is_slow: slow %d", slow); + + return slow; +} + +/* + * To be deemed as soft real-time, an application must meet two + * requirements. First, the application must not require an average + * bandwidth higher than the approximate bandwidth required to playback or + * record a compressed high-definition video. + * The next function is invoked on the completion of the last request of a + * batch, to compute the next-start time instant, soft_rt_next_start, such + * that, if the next request of the application does not arrive before + * soft_rt_next_start, then the above requirement on the bandwidth is met. + * + * The second requirement is that the request pattern of the application is + * isochronous, i.e., that, after issuing a request or a batch of requests, + * the application stops issuing new requests until all its pending requests + * have been completed. After that, the application may issue a new batch, + * and so on. + * For this reason the next function is invoked to compute + * soft_rt_next_start only for applications that meet this requirement, + * whereas soft_rt_next_start is set to infinity for applications that do + * not. + * + * Unfortunately, even a greedy application may happen to behave in an + * isochronous way if the CPU load is high. In fact, the application may + * stop issuing requests while the CPUs are busy serving other processes, + * then restart, then stop again for a while, and so on. In addition, if + * the disk achieves a low enough throughput with the request pattern + * issued by the application (e.g., because the request pattern is random + * and/or the device is slow), then the application may meet the above + * bandwidth requirement too. To prevent such a greedy application to be + * deemed as soft real-time, a further rule is used in the computation of + * soft_rt_next_start: soft_rt_next_start must be higher than the current + * time plus the maximum time for which the arrival of a request is waited + * for when a sync queue becomes idle, namely bfqd->bfq_slice_idle. + * This filters out greedy applications, as the latter issue instead their + * next request as soon as possible after the last one has been completed + * (in contrast, when a batch of requests is completed, a soft real-time + * application spends some time processing data). + * + * Unfortunately, the last filter may easily generate false positives if + * only bfqd->bfq_slice_idle is used as a reference time interval and one + * or both the following cases occur: + * 1) HZ is so low that the duration of a jiffy is comparable to or higher + * than bfqd->bfq_slice_idle. This happens, e.g., on slow devices with + * HZ=100. + * 2) jiffies, instead of increasing at a constant rate, may stop increasing + * for a while, then suddenly 'jump' by several units to recover the lost + * increments. This seems to happen, e.g., inside virtual machines. + * To address this issue, we do not use as a reference time interval just + * bfqd->bfq_slice_idle, but bfqd->bfq_slice_idle plus a few jiffies. In + * particular we add the minimum number of jiffies for which the filter + * seems to be quite precise also in embedded systems and KVM/QEMU virtual + * machines. + */ +static unsigned long bfq_bfqq_softrt_next_start(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqd, bfqq, +"softrt_next_start: service_blkg %lu soft_rate %u sects/sec interval %u", + bfqq->service_from_backlogged, + bfqd->bfq_wr_max_softrt_rate, + jiffies_to_msecs(HZ * bfqq->service_from_backlogged / + bfqd->bfq_wr_max_softrt_rate)); + + return max(bfqq->last_idle_bklogged + + HZ * bfqq->service_from_backlogged / + bfqd->bfq_wr_max_softrt_rate, + jiffies + nsecs_to_jiffies(bfqq->bfqd->bfq_slice_idle) + 4); +} + +/** + * bfq_bfqq_expire - expire a queue. + * @bfqd: device owning the queue. + * @bfqq: the queue to expire. + * @compensate: if true, compensate for the time spent idling. + * @reason: the reason causing the expiration. + * + * If the process associated with bfqq does slow I/O (e.g., because it + * issues random requests), we charge bfqq with the time it has been + * in service instead of the service it has received (see + * bfq_bfqq_charge_time for details on how this goal is achieved). As + * a consequence, bfqq will typically get higher timestamps upon + * reactivation, and hence it will be rescheduled as if it had + * received more service than what it has actually received. In the + * end, bfqq receives less service in proportion to how slowly its + * associated process consumes its budgets (and hence how seriously it + * tends to lower the throughput). In addition, this time-charging + * strategy guarantees time fairness among slow processes. In + * contrast, if the process associated with bfqq is not slow, we + * charge bfqq exactly with the service it has received. + * + * Charging time to the first type of queues and the exact service to + * the other has the effect of using the WF2Q+ policy to schedule the + * former on a timeslice basis, without violating service domain + * guarantees among the latter. + */ +static void bfq_bfqq_expire(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + bool compensate, + enum bfqq_expiration reason) +{ + bool slow; + unsigned long delta = 0; + struct bfq_entity *entity = &bfqq->entity; + int ref; + + BUG_ON(bfqq != bfqd->in_service_queue); + + /* + * Check whether the process is slow (see bfq_bfqq_is_slow). + */ + slow = bfq_bfqq_is_slow(bfqd, bfqq, compensate, reason, &delta); + + /* + * Increase service_from_backlogged before next statement, + * because the possible next invocation of + * bfq_bfqq_charge_time would likely inflate + * entity->service. In contrast, service_from_backlogged must + * contain real service, to enable the soft real-time + * heuristic to correctly compute the bandwidth consumed by + * bfqq. + */ + bfqq->service_from_backlogged += entity->service; + + /* + * As above explained, charge slow (typically seeky) and + * timed-out queues with the time and not the service + * received, to favor sequential workloads. + * + * Processes doing I/O in the slower disk zones will tend to + * be slow(er) even if not seeky. Therefore, since the + * estimated peak rate is actually an average over the disk + * surface, these processes may timeout just for bad luck. To + * avoid punishing them, do not charge time to processes that + * succeeded in consuming at least 2/3 of their budget. This + * allows BFQ to preserve enough elasticity to still perform + * bandwidth, and not time, distribution with little unlucky + * or quasi-sequential processes. + */ + if (bfqq->wr_coeff == 1 && + (slow || + (reason == BFQ_BFQQ_BUDGET_TIMEOUT && + bfq_bfqq_budget_left(bfqq) >= entity->budget / 3))) + bfq_bfqq_charge_time(bfqd, bfqq, delta); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + if (reason == BFQ_BFQQ_TOO_IDLE && + entity->service <= 2 * entity->budget / 10) + bfq_clear_bfqq_IO_bound(bfqq); + + if (bfqd->low_latency && bfqq->wr_coeff == 1) + bfqq->last_wr_start_finish = jiffies; + + if (bfqd->low_latency && bfqd->bfq_wr_max_softrt_rate > 0 && + RB_EMPTY_ROOT(&bfqq->sort_list)) { + /* + * If we get here, and there are no outstanding + * requests, then the request pattern is isochronous + * (see the comments on the function + * bfq_bfqq_softrt_next_start()). Thus we can compute + * soft_rt_next_start. If, instead, the queue still + * has outstanding requests, then we have to wait for + * the completion of all the outstanding requests to + * discover whether the request pattern is actually + * isochronous. + */ + BUG_ON(bfqd->busy_queues < 1); + if (bfqq->dispatched == 0) { + bfqq->soft_rt_next_start = + bfq_bfqq_softrt_next_start(bfqd, bfqq); + bfq_log_bfqq(bfqd, bfqq, "new soft_rt_next %lu", + bfqq->soft_rt_next_start); + } else { + /* + * The application is still waiting for the + * completion of one or more requests: + * prevent it from possibly being incorrectly + * deemed as soft real-time by setting its + * soft_rt_next_start to infinity. In fact, + * without this assignment, the application + * would be incorrectly deemed as soft + * real-time if: + * 1) it issued a new request before the + * completion of all its in-flight + * requests, and + * 2) at that time, its soft_rt_next_start + * happened to be in the past. + */ + bfqq->soft_rt_next_start = + bfq_greatest_from_now(); + /* + * Schedule an update of soft_rt_next_start to when + * the task may be discovered to be isochronous. + */ + bfq_mark_bfqq_softrt_update(bfqq); + } + } + + bfq_log_bfqq(bfqd, bfqq, + "expire (%d, slow %d, num_disp %d, short_ttime %d, weight %d)", + reason, slow, bfqq->dispatched, + bfq_bfqq_has_short_ttime(bfqq), entity->weight); + + /* + * Increase, decrease or leave budget unchanged according to + * reason. + */ + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + __bfq_bfqq_recalc_budget(bfqd, bfqq, reason); + BUG_ON(bfqq->next_rq == NULL && + bfqq->entity.budget < bfqq->entity.service); + ref = bfqq->ref; + __bfq_bfqq_expire(bfqd, bfqq); + + BUG_ON(ref > 1 && + !bfq_bfqq_busy(bfqq) && reason == BFQ_BFQQ_BUDGET_EXHAUSTED && + !bfq_class_idle(bfqq)); + + /* mark bfqq as waiting a request only if a bic still points to it */ + if (ref > 1 && !bfq_bfqq_busy(bfqq) && + reason != BFQ_BFQQ_BUDGET_TIMEOUT && + reason != BFQ_BFQQ_BUDGET_EXHAUSTED) + bfq_mark_bfqq_non_blocking_wait_rq(bfqq); +} + +/* + * Budget timeout is not implemented through a dedicated timer, but + * just checked on request arrivals and completions, as well as on + * idle timer expirations. + */ +static bool bfq_bfqq_budget_timeout(struct bfq_queue *bfqq) +{ + return time_is_before_eq_jiffies(bfqq->budget_timeout); +} + +/* + * If we expire a queue that is actively waiting (i.e., with the + * device idled) for the arrival of a new request, then we may incur + * the timestamp misalignment problem described in the body of the + * function __bfq_activate_entity. Hence we return true only if this + * condition does not hold, or if the queue is slow enough to deserve + * only to be kicked off for preserving a high throughput. + */ +static bool bfq_may_expire_for_budg_timeout(struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqq->bfqd, bfqq, + "may_budget_timeout: wait_request %d left %d timeout %d", + bfq_bfqq_wait_request(bfqq), + bfq_bfqq_budget_left(bfqq) >= bfqq->entity.budget / 3, + bfq_bfqq_budget_timeout(bfqq)); + + return (!bfq_bfqq_wait_request(bfqq) || + bfq_bfqq_budget_left(bfqq) >= bfqq->entity.budget / 3) + && + bfq_bfqq_budget_timeout(bfqq); +} + +/* + * For a queue that becomes empty, device idling is allowed only if + * this function returns true for that queue. As a consequence, since + * device idling plays a critical role for both throughput boosting + * and service guarantees, the return value of this function plays a + * critical role as well. + * + * In a nutshell, this function returns true only if idling is + * beneficial for throughput or, even if detrimental for throughput, + * idling is however necessary to preserve service guarantees (low + * latency, desired throughput distribution, ...). In particular, on + * NCQ-capable devices, this function tries to return false, so as to + * help keep the drives' internal queues full, whenever this helps the + * device boost the throughput without causing any service-guarantee + * issue. + * + * In more detail, the return value of this function is obtained by, + * first, computing a number of boolean variables that take into + * account throughput and service-guarantee issues, and, then, + * combining these variables in a logical expression. Most of the + * issues taken into account are not trivial. We discuss these issues + * while introducing the variables. + */ +static bool bfq_bfqq_may_idle(struct bfq_queue *bfqq) +{ + struct bfq_data *bfqd = bfqq->bfqd; + bool rot_without_queueing = + !blk_queue_nonrot(bfqd->queue) && !bfqd->hw_tag, + bfqq_sequential_and_IO_bound, + idling_boosts_thr, idling_boosts_thr_without_issues, + idling_needed_for_service_guarantees, + asymmetric_scenario; + + if (bfqd->strict_guarantees) + return true; + + /* + * Idling is performed only if slice_idle > 0. In addition, we + * do not idle if + * (a) bfqq is async + * (b) bfqq is in the idle io prio class: in this case we do + * not idle because we want to minimize the bandwidth that + * queues in this class can steal to higher-priority queues + */ + if (bfqd->bfq_slice_idle == 0 || !bfq_bfqq_sync(bfqq) || + bfq_class_idle(bfqq)) + return false; + + bfqq_sequential_and_IO_bound = !BFQQ_SEEKY(bfqq) && + bfq_bfqq_IO_bound(bfqq) && bfq_bfqq_has_short_ttime(bfqq); + /* + * The next variable takes into account the cases where idling + * boosts the throughput. + * + * The value of the variable is computed considering, first, that + * idling is virtually always beneficial for the throughput if: + * (a) the device is not NCQ-capable and rotational, or + * (b) regardless of the presence of NCQ, the device is rotational and + * the request pattern for bfqq is I/O-bound and sequential, or + * (c) regardless of whether it is rotational, the device is + * not NCQ-capable and the request pattern for bfqq is + * I/O-bound and sequential. + * + * Secondly, and in contrast to the above item (b), idling an + * NCQ-capable flash-based device would not boost the + * throughput even with sequential I/O; rather it would lower + * the throughput in proportion to how fast the device + * is. Accordingly, the next variable is true if any of the + * above conditions (a), (b) or (c) is true, and, in + * particular, happens to be false if bfqd is an NCQ-capable + * flash-based device. + */ + idling_boosts_thr = rot_without_queueing || + ((!blk_queue_nonrot(bfqd->queue) || !bfqd->hw_tag) && + bfqq_sequential_and_IO_bound); + + /* + * The value of the next variable, + * idling_boosts_thr_without_issues, is equal to that of + * idling_boosts_thr, unless a special case holds. In this + * special case, described below, idling may cause problems to + * weight-raised queues. + * + * When the request pool is saturated (e.g., in the presence + * of write hogs), if the processes associated with + * non-weight-raised queues ask for requests at a lower rate, + * then processes associated with weight-raised queues have a + * higher probability to get a request from the pool + * immediately (or at least soon) when they need one. Thus + * they have a higher probability to actually get a fraction + * of the device throughput proportional to their high + * weight. This is especially true with NCQ-capable drives, + * which enqueue several requests in advance, and further + * reorder internally-queued requests. + * + * For this reason, we force to false the value of + * idling_boosts_thr_without_issues if there are weight-raised + * busy queues. In this case, and if bfqq is not weight-raised, + * this guarantees that the device is not idled for bfqq (if, + * instead, bfqq is weight-raised, then idling will be + * guaranteed by another variable, see below). Combined with + * the timestamping rules of BFQ (see [1] for details), this + * behavior causes bfqq, and hence any sync non-weight-raised + * queue, to get a lower number of requests served, and thus + * to ask for a lower number of requests from the request + * pool, before the busy weight-raised queues get served + * again. This often mitigates starvation problems in the + * presence of heavy write workloads and NCQ, thereby + * guaranteeing a higher application and system responsiveness + * in these hostile scenarios. + */ + idling_boosts_thr_without_issues = idling_boosts_thr && + bfqd->wr_busy_queues == 0; + + /* + * There is then a case where idling must be performed not + * for throughput concerns, but to preserve service + * guarantees. + * + * To introduce this case, we can note that allowing the drive + * to enqueue more than one request at a time, and hence + * delegating de facto final scheduling decisions to the + * drive's internal scheduler, entails loss of control on the + * actual request service order. In particular, the critical + * situation is when requests from different processes happen + * to be present, at the same time, in the internal queue(s) + * of the drive. In such a situation, the drive, by deciding + * the service order of the internally-queued requests, does + * determine also the actual throughput distribution among + * these processes. But the drive typically has no notion or + * concern about per-process throughput distribution, and + * makes its decisions only on a per-request basis. Therefore, + * the service distribution enforced by the drive's internal + * scheduler is likely to coincide with the desired + * device-throughput distribution only in a completely + * symmetric scenario where: + * (i) each of these processes must get the same throughput as + * the others; + * (ii) all these processes have the same I/O pattern + * (either sequential or random). + * In fact, in such a scenario, the drive will tend to treat + * the requests of each of these processes in about the same + * way as the requests of the others, and thus to provide + * each of these processes with about the same throughput + * (which is exactly the desired throughput distribution). In + * contrast, in any asymmetric scenario, device idling is + * certainly needed to guarantee that bfqq receives its + * assigned fraction of the device throughput (see [1] for + * details). + * + * We address this issue by controlling, actually, only the + * symmetry sub-condition (i), i.e., provided that + * sub-condition (i) holds, idling is not performed, + * regardless of whether sub-condition (ii) holds. In other + * words, only if sub-condition (i) holds, then idling is + * allowed, and the device tends to be prevented from queueing + * many requests, possibly of several processes. The reason + * for not controlling also sub-condition (ii) is that we + * exploit preemption to preserve guarantees in case of + * symmetric scenarios, even if (ii) does not hold, as + * explained in the next two paragraphs. + * + * Even if a queue, say Q, is expired when it remains idle, Q + * can still preempt the new in-service queue if the next + * request of Q arrives soon (see the comments on + * bfq_bfqq_update_budg_for_activation). If all queues and + * groups have the same weight, this form of preemption, + * combined with the hole-recovery heuristic described in the + * comments on function bfq_bfqq_update_budg_for_activation, + * are enough to preserve a correct bandwidth distribution in + * the mid term, even without idling. In fact, even if not + * idling allows the internal queues of the device to contain + * many requests, and thus to reorder requests, we can rather + * safely assume that the internal scheduler still preserves a + * minimum of mid-term fairness. The motivation for using + * preemption instead of idling is that, by not idling, + * service guarantees are preserved without minimally + * sacrificing throughput. In other words, both a high + * throughput and its desired distribution are obtained. + * + * More precisely, this preemption-based, idleless approach + * provides fairness in terms of IOPS, and not sectors per + * second. This can be seen with a simple example. Suppose + * that there are two queues with the same weight, but that + * the first queue receives requests of 8 sectors, while the + * second queue receives requests of 1024 sectors. In + * addition, suppose that each of the two queues contains at + * most one request at a time, which implies that each queue + * always remains idle after it is served. Finally, after + * remaining idle, each queue receives very quickly a new + * request. It follows that the two queues are served + * alternatively, preempting each other if needed. This + * implies that, although both queues have the same weight, + * the queue with large requests receives a service that is + * 1024/8 times as high as the service received by the other + * queue. + * + * On the other hand, device idling is performed, and thus + * pure sector-domain guarantees are provided, for the + * following queues, which are likely to need stronger + * throughput guarantees: weight-raised queues, and queues + * with a higher weight than other queues. When such queues + * are active, sub-condition (i) is false, which triggers + * device idling. + * + * According to the above considerations, the next variable is + * true (only) if sub-condition (i) holds. To compute the + * value of this variable, we not only use the return value of + * the function bfq_symmetric_scenario(), but also check + * whether bfqq is being weight-raised, because + * bfq_symmetric_scenario() does not take into account also + * weight-raised queues (see comments on + * bfq_weights_tree_add()). + * + * As a side note, it is worth considering that the above + * device-idling countermeasures may however fail in the + * following unlucky scenario: if idling is (correctly) + * disabled in a time period during which all symmetry + * sub-conditions hold, and hence the device is allowed to + * enqueue many requests, but at some later point in time some + * sub-condition stops to hold, then it may become impossible + * to let requests be served in the desired order until all + * the requests already queued in the device have been served. + */ + asymmetric_scenario = bfqq->wr_coeff > 1 || + !bfq_symmetric_scenario(bfqd); + + /* + * Finally, there is a case where maximizing throughput is the + * best choice even if it may cause unfairness toward + * bfqq. Such a case is when bfqq became active in a burst of + * queue activations. Queues that became active during a large + * burst benefit only from throughput, as discussed in the + * comments on bfq_handle_burst. Thus, if bfqq became active + * in a burst and not idling the device maximizes throughput, + * then the device must no be idled, because not idling the + * device provides bfqq and all other queues in the burst with + * maximum benefit. Combining this and the above case, we can + * now establish when idling is actually needed to preserve + * service guarantees. + */ + idling_needed_for_service_guarantees = + asymmetric_scenario && !bfq_bfqq_in_large_burst(bfqq); + + /* + * We have now all the components we need to compute the + * return value of the function, which is true only if idling + * either boosts the throughput (without issues), or is + * necessary to preserve service guarantees. + */ + bfq_log_bfqq(bfqd, bfqq, "may_idle: sync %d idling_boosts_thr %d", + bfq_bfqq_sync(bfqq), idling_boosts_thr); + + bfq_log_bfqq(bfqd, bfqq, + "may_idle: wr_busy %d boosts %d IO-bound %d guar %d", + bfqd->wr_busy_queues, + idling_boosts_thr_without_issues, + bfq_bfqq_IO_bound(bfqq), + idling_needed_for_service_guarantees); + + return idling_boosts_thr_without_issues || + idling_needed_for_service_guarantees; +} + +/* + * If the in-service queue is empty but the function bfq_bfqq_may_idle + * returns true, then: + * 1) the queue must remain in service and cannot be expired, and + * 2) the device must be idled to wait for the possible arrival of a new + * request for the queue. + * See the comments on the function bfq_bfqq_may_idle for the reasons + * why performing device idling is the best choice to boost the throughput + * and preserve service guarantees when bfq_bfqq_may_idle itself + * returns true. + */ +static bool bfq_bfqq_must_idle(struct bfq_queue *bfqq) +{ + return RB_EMPTY_ROOT(&bfqq->sort_list) && bfq_bfqq_may_idle(bfqq); +} + +/* + * Select a queue for service. If we have a current queue in service, + * check whether to continue servicing it, or retrieve and set a new one. + */ +static struct bfq_queue *bfq_select_queue(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq; + struct request *next_rq; + enum bfqq_expiration reason = BFQ_BFQQ_BUDGET_TIMEOUT; + + bfqq = bfqd->in_service_queue; + if (!bfqq) + goto new_queue; + + bfq_log_bfqq(bfqd, bfqq, "select_queue: already in-service queue"); + + if (bfq_may_expire_for_budg_timeout(bfqq) && + !hrtimer_active(&bfqd->idle_slice_timer) && + !bfq_bfqq_must_idle(bfqq)) + goto expire; + +check_queue: + /* + * This loop is rarely executed more than once. Even when it + * happens, it is much more convenient to re-execute this loop + * than to return NULL and trigger a new dispatch to get a + * request served. + */ + next_rq = bfqq->next_rq; + /* + * If bfqq has requests queued and it has enough budget left to + * serve them, keep the queue, otherwise expire it. + */ + if (next_rq) { + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + + if (bfq_serv_to_charge(next_rq, bfqq) > + bfq_bfqq_budget_left(bfqq)) { + /* + * Expire the queue for budget exhaustion, + * which makes sure that the next budget is + * enough to serve the next request, even if + * it comes from the fifo expired path. + */ + reason = BFQ_BFQQ_BUDGET_EXHAUSTED; + goto expire; + } else { + /* + * The idle timer may be pending because we may + * not disable disk idling even when a new request + * arrives. + */ + if (bfq_bfqq_wait_request(bfqq)) { + BUG_ON(!hrtimer_active(&bfqd->idle_slice_timer)); + /* + * If we get here: 1) at least a new request + * has arrived but we have not disabled the + * timer because the request was too small, + * 2) then the block layer has unplugged + * the device, causing the dispatch to be + * invoked. + * + * Since the device is unplugged, now the + * requests are probably large enough to + * provide a reasonable throughput. + * So we disable idling. + */ + bfq_clear_bfqq_wait_request(bfqq); + hrtimer_try_to_cancel(&bfqd->idle_slice_timer); + bfqg_stats_update_idle_time(bfqq_group(bfqq)); + } + goto keep_queue; + } + } + + /* + * No requests pending. However, if the in-service queue is idling + * for a new request, or has requests waiting for a completion and + * may idle after their completion, then keep it anyway. + */ + if (hrtimer_active(&bfqd->idle_slice_timer) || + (bfqq->dispatched != 0 && bfq_bfqq_may_idle(bfqq))) { + bfqq = NULL; + goto keep_queue; + } + + reason = BFQ_BFQQ_NO_MORE_REQUESTS; +expire: + bfq_bfqq_expire(bfqd, bfqq, false, reason); +new_queue: + bfqq = bfq_set_in_service_queue(bfqd); + if (bfqq) { + bfq_log_bfqq(bfqd, bfqq, "select_queue: checking new queue"); + goto check_queue; + } +keep_queue: + if (bfqq) + bfq_log_bfqq(bfqd, bfqq, "select_queue: returned this queue"); + else + bfq_log(bfqd, "select_queue: no queue returned"); + + return bfqq; +} + +static void bfq_update_wr_data(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + struct bfq_entity *entity = &bfqq->entity; + + if (bfqq->wr_coeff > 1) { /* queue is being weight-raised */ + BUG_ON(bfqq->wr_cur_max_time == bfqd->bfq_wr_rt_max_time && + time_is_after_jiffies(bfqq->last_wr_start_finish)); + + bfq_log_bfqq(bfqd, bfqq, + "raising period dur %u/%u msec, old coeff %u, w %d(%d)", + jiffies_to_msecs(jiffies - bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time), + bfqq->wr_coeff, + bfqq->entity.weight, bfqq->entity.orig_weight); + + BUG_ON(bfqq != bfqd->in_service_queue && entity->weight != + entity->orig_weight * bfqq->wr_coeff); + if (entity->prio_changed) + bfq_log_bfqq(bfqd, bfqq, "WARN: pending prio change"); + + /* + * If the queue was activated in a burst, or too much + * time has elapsed from the beginning of this + * weight-raising period, then end weight raising. + */ + if (bfq_bfqq_in_large_burst(bfqq)) + bfq_bfqq_end_wr(bfqq); + else if (time_is_before_jiffies(bfqq->last_wr_start_finish + + bfqq->wr_cur_max_time)) { + if (bfqq->wr_cur_max_time != bfqd->bfq_wr_rt_max_time || + time_is_before_jiffies(bfqq->wr_start_at_switch_to_srt + + bfq_wr_duration(bfqd))) + bfq_bfqq_end_wr(bfqq); + else { + /* switch back to interactive wr */ + bfqq->wr_coeff = bfqd->bfq_wr_coeff; + bfqq->wr_cur_max_time = bfq_wr_duration(bfqd); + bfqq->last_wr_start_finish = + bfqq->wr_start_at_switch_to_srt; + BUG_ON(time_is_after_jiffies( + bfqq->last_wr_start_finish)); + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqd, bfqq, + "back to interactive wr"); + } + } + } + /* + * To improve latency (for this or other queues), immediately + * update weight both if it must be raised and if it must be + * lowered. Since, entity may be on some active tree here, and + * might have a pending change of its ioprio class, invoke + * next function with the last parameter unset (see the + * comments on the function). + */ + if ((entity->weight > entity->orig_weight) != (bfqq->wr_coeff > 1)) + __bfq_entity_update_weight_prio(bfq_entity_service_tree(entity), + entity, false); +} + +/* + * Dispatch one request from bfqq, moving it to the request queue + * dispatch list. + */ +static int bfq_dispatch_request(struct bfq_data *bfqd, + struct bfq_queue *bfqq) +{ + int dispatched = 0; + struct request *rq = bfqq->next_rq; + unsigned long service_to_charge; + + BUG_ON(RB_EMPTY_ROOT(&bfqq->sort_list)); + BUG_ON(!rq); + service_to_charge = bfq_serv_to_charge(rq, bfqq); + + BUG_ON(service_to_charge > bfq_bfqq_budget_left(bfqq)); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + bfq_bfqq_served(bfqq, service_to_charge); + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + bfq_dispatch_insert(bfqd->queue, rq); + + /* + * If weight raising has to terminate for bfqq, then next + * function causes an immediate update of bfqq's weight, + * without waiting for next activation. As a consequence, on + * expiration, bfqq will be timestamped as if has never been + * weight-raised during this service slot, even if it has + * received part or even most of the service as a + * weight-raised queue. This inflates bfqq's timestamps, which + * is beneficial, as bfqq is then more willing to leave the + * device immediately to possible other weight-raised queues. + */ + bfq_update_wr_data(bfqd, bfqq); + + bfq_log_bfqq(bfqd, bfqq, + "dispatched %u sec req (%llu), budg left %d", + blk_rq_sectors(rq), + (unsigned long long) blk_rq_pos(rq), + bfq_bfqq_budget_left(bfqq)); + + dispatched++; + + if (!bfqd->in_service_bic) { + atomic_long_inc(&RQ_BIC(rq)->icq.ioc->refcount); + bfqd->in_service_bic = RQ_BIC(rq); + BUG_ON(!bfqd->in_service_bic); + } + + if (bfqd->busy_queues > 1 && bfq_class_idle(bfqq)) + goto expire; + + return dispatched; + +expire: + bfq_bfqq_expire(bfqd, bfqq, false, BFQ_BFQQ_BUDGET_EXHAUSTED); + return dispatched; +} + +static int __bfq_forced_dispatch_bfqq(struct bfq_queue *bfqq) +{ + int dispatched = 0; + + while (bfqq->next_rq) { + bfq_dispatch_insert(bfqq->bfqd->queue, bfqq->next_rq); + dispatched++; + } + + BUG_ON(!list_empty(&bfqq->fifo)); + return dispatched; +} + +/* + * Drain our current requests. + * Used for barriers and when switching io schedulers on-the-fly. + */ +static int bfq_forced_dispatch(struct bfq_data *bfqd) +{ + struct bfq_queue *bfqq, *n; + struct bfq_service_tree *st; + int dispatched = 0; + + bfqq = bfqd->in_service_queue; + if (bfqq) + __bfq_bfqq_expire(bfqd, bfqq); + + /* + * Loop through classes, and be careful to leave the scheduler + * in a consistent state, as feedback mechanisms and vtime + * updates cannot be disabled during the process. + */ + list_for_each_entry_safe(bfqq, n, &bfqd->active_list, bfqq_list) { + st = bfq_entity_service_tree(&bfqq->entity); + + dispatched += __bfq_forced_dispatch_bfqq(bfqq); + + bfqq->max_budget = bfq_max_budget(bfqd); + bfq_forget_idle(st); + } + + BUG_ON(bfqd->busy_queues != 0); + + return dispatched; +} + +static int bfq_dispatch_requests(struct request_queue *q, int force) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + struct bfq_queue *bfqq; + + bfq_log(bfqd, "dispatch requests: %d busy queues", bfqd->busy_queues); + + if (bfqd->busy_queues == 0) + return 0; + + if (unlikely(force)) + return bfq_forced_dispatch(bfqd); + + /* + * Force device to serve one request at a time if + * strict_guarantees is true. Forcing this service scheme is + * currently the ONLY way to guarantee that the request + * service order enforced by the scheduler is respected by a + * queueing device. Otherwise the device is free even to make + * some unlucky request wait for as long as the device + * wishes. + * + * Of course, serving one request at at time may cause loss of + * throughput. + */ + if (bfqd->strict_guarantees && bfqd->rq_in_driver > 0) + return 0; + + bfqq = bfq_select_queue(bfqd); + if (!bfqq) + return 0; + + BUG_ON(bfqq->entity.budget < bfqq->entity.service); + + BUG_ON(bfq_bfqq_wait_request(bfqq)); + + if (!bfq_dispatch_request(bfqd, bfqq)) + return 0; + + bfq_log_bfqq(bfqd, bfqq, "dispatched %s request", + bfq_bfqq_sync(bfqq) ? "sync" : "async"); + + BUG_ON(bfqq->next_rq == NULL && + bfqq->entity.budget < bfqq->entity.service); + return 1; +} + +/* + * Task holds one reference to the queue, dropped when task exits. Each rq + * in-flight on this queue also holds a reference, dropped when rq is freed. + * + * Queue lock must be held here. Recall not to use bfqq after calling + * this function on it. + */ +static void bfq_put_queue(struct bfq_queue *bfqq) +{ +#ifdef BFQ_GROUP_IOSCHED_ENABLED + struct bfq_group *bfqg = bfqq_group(bfqq); +#endif + + BUG_ON(bfqq->ref <= 0); + + bfq_log_bfqq(bfqq->bfqd, bfqq, "put_queue: %p %d", bfqq, bfqq->ref); + bfqq->ref--; + if (bfqq->ref) + return; + + BUG_ON(rb_first(&bfqq->sort_list)); + BUG_ON(bfqq->allocated[READ] + bfqq->allocated[WRITE] != 0); + BUG_ON(bfqq->entity.tree); + BUG_ON(bfq_bfqq_busy(bfqq)); + + if (bfq_bfqq_sync(bfqq)) + /* + * The fact that this queue is being destroyed does not + * invalidate the fact that this queue may have been + * activated during the current burst. As a consequence, + * although the queue does not exist anymore, and hence + * needs to be removed from the burst list if there, + * the burst size has not to be decremented. + */ + hlist_del_init(&bfqq->burst_list_node); + + bfq_log_bfqq(bfqq->bfqd, bfqq, "put_queue: %p freed", bfqq); + + kmem_cache_free(bfq_pool, bfqq); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + bfqg_put(bfqg); +#endif +} + +static void bfq_put_cooperator(struct bfq_queue *bfqq) +{ + struct bfq_queue *__bfqq, *next; + + /* + * If this queue was scheduled to merge with another queue, be + * sure to drop the reference taken on that queue (and others in + * the merge chain). See bfq_setup_merge and bfq_merge_bfqqs. + */ + __bfqq = bfqq->new_bfqq; + while (__bfqq) { + if (__bfqq == bfqq) + break; + next = __bfqq->new_bfqq; + bfq_put_queue(__bfqq); + __bfqq = next; + } +} + +static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq) +{ + if (bfqq == bfqd->in_service_queue) { + __bfq_bfqq_expire(bfqd, bfqq); + bfq_schedule_dispatch(bfqd); + } + + bfq_log_bfqq(bfqd, bfqq, "exit_bfqq: %p, %d", bfqq, bfqq->ref); + + bfq_put_cooperator(bfqq); + + bfq_put_queue(bfqq); /* release process reference */ +} + +static void bfq_init_icq(struct io_cq *icq) +{ + icq_to_bic(icq)->ttime.last_end_request = ktime_get_ns() - (1ULL<<32); +} + +static void bfq_exit_icq(struct io_cq *icq) +{ + struct bfq_io_cq *bic = icq_to_bic(icq); + struct bfq_data *bfqd = bic_to_bfqd(bic); + + if (bic_to_bfqq(bic, false)) { + bfq_exit_bfqq(bfqd, bic_to_bfqq(bic, false)); + bic_set_bfqq(bic, NULL, false); + } + + if (bic_to_bfqq(bic, true)) { + /* + * If the bic is using a shared queue, put the reference + * taken on the io_context when the bic started using a + * shared bfq_queue. + */ + if (bfq_bfqq_coop(bic_to_bfqq(bic, true))) + put_io_context(icq->ioc); + bfq_exit_bfqq(bfqd, bic_to_bfqq(bic, true)); + bic_set_bfqq(bic, NULL, true); + } +} + +/* + * Update the entity prio values; note that the new values will not + * be used until the next (re)activation. + */ +static void bfq_set_next_ioprio_data(struct bfq_queue *bfqq, + struct bfq_io_cq *bic) +{ + struct task_struct *tsk = current; + int ioprio_class; + + ioprio_class = IOPRIO_PRIO_CLASS(bic->ioprio); + switch (ioprio_class) { + default: + dev_err(bfqq->bfqd->queue->backing_dev_info->dev, + "bfq: bad prio class %d\n", ioprio_class); + case IOPRIO_CLASS_NONE: + /* + * No prio set, inherit CPU scheduling settings. + */ + bfqq->new_ioprio = task_nice_ioprio(tsk); + bfqq->new_ioprio_class = task_nice_ioclass(tsk); + break; + case IOPRIO_CLASS_RT: + bfqq->new_ioprio = IOPRIO_PRIO_DATA(bic->ioprio); + bfqq->new_ioprio_class = IOPRIO_CLASS_RT; + break; + case IOPRIO_CLASS_BE: + bfqq->new_ioprio = IOPRIO_PRIO_DATA(bic->ioprio); + bfqq->new_ioprio_class = IOPRIO_CLASS_BE; + break; + case IOPRIO_CLASS_IDLE: + bfqq->new_ioprio_class = IOPRIO_CLASS_IDLE; + bfqq->new_ioprio = 7; + break; + } + + if (bfqq->new_ioprio >= IOPRIO_BE_NR) { + pr_crit("bfq_set_next_ioprio_data: new_ioprio %d\n", + bfqq->new_ioprio); + BUG(); + } + + bfqq->entity.new_weight = bfq_ioprio_to_weight(bfqq->new_ioprio); + bfqq->entity.prio_changed = 1; + bfq_log_bfqq(bfqq->bfqd, bfqq, + "set_next_ioprio_data: bic_class %d prio %d class %d", + ioprio_class, bfqq->new_ioprio, bfqq->new_ioprio_class); +} + +static void bfq_check_ioprio_change(struct bfq_io_cq *bic, struct bio *bio) +{ + struct bfq_data *bfqd = bic_to_bfqd(bic); + struct bfq_queue *bfqq; + unsigned long uninitialized_var(flags); + int ioprio = bic->icq.ioc->ioprio; + + /* + * This condition may trigger on a newly created bic, be sure to + * drop the lock before returning. + */ + if (unlikely(!bfqd) || likely(bic->ioprio == ioprio)) + return; + + bic->ioprio = ioprio; + + bfqq = bic_to_bfqq(bic, false); + if (bfqq) { + /* release process reference on this queue */ + bfq_put_queue(bfqq); + bfqq = bfq_get_queue(bfqd, bio, BLK_RW_ASYNC, bic); + bic_set_bfqq(bic, bfqq, false); + bfq_log_bfqq(bfqd, bfqq, + "check_ioprio_change: bfqq %p %d", + bfqq, bfqq->ref); + } + + bfqq = bic_to_bfqq(bic, true); + if (bfqq) + bfq_set_next_ioprio_data(bfqq, bic); +} + +static void bfq_init_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct bfq_io_cq *bic, pid_t pid, int is_sync) +{ + RB_CLEAR_NODE(&bfqq->entity.rb_node); + INIT_LIST_HEAD(&bfqq->fifo); + INIT_HLIST_NODE(&bfqq->burst_list_node); + BUG_ON(!hlist_unhashed(&bfqq->burst_list_node)); + + bfqq->ref = 0; + bfqq->bfqd = bfqd; + + if (bic) + bfq_set_next_ioprio_data(bfqq, bic); + + if (is_sync) { + /* + * No need to mark as has_short_ttime if in + * idle_class, because no device idling is performed + * for queues in idle class + */ + if (!bfq_class_idle(bfqq)) + /* tentatively mark as has_short_ttime */ + bfq_mark_bfqq_has_short_ttime(bfqq); + bfq_mark_bfqq_sync(bfqq); + bfq_mark_bfqq_just_created(bfqq); + } else + bfq_clear_bfqq_sync(bfqq); + bfq_mark_bfqq_IO_bound(bfqq); + + /* Tentative initial value to trade off between thr and lat */ + bfqq->max_budget = (2 * bfq_max_budget(bfqd)) / 3; + bfqq->pid = pid; + + bfqq->wr_coeff = 1; + bfqq->last_wr_start_finish = jiffies; + bfqq->wr_start_at_switch_to_srt = bfq_smallest_from_now(); + bfqq->budget_timeout = bfq_smallest_from_now(); + bfqq->split_time = bfq_smallest_from_now(); + + /* + * Set to the value for which bfqq will not be deemed as + * soft rt when it becomes backlogged. + */ + bfqq->soft_rt_next_start = bfq_greatest_from_now(); + + /* first request is almost certainly seeky */ + bfqq->seek_history = 1; +} + +static struct bfq_queue **bfq_async_queue_prio(struct bfq_data *bfqd, + struct bfq_group *bfqg, + int ioprio_class, int ioprio) +{ + switch (ioprio_class) { + case IOPRIO_CLASS_RT: + return &bfqg->async_bfqq[0][ioprio]; + case IOPRIO_CLASS_NONE: + ioprio = IOPRIO_NORM; + /* fall through */ + case IOPRIO_CLASS_BE: + return &bfqg->async_bfqq[1][ioprio]; + case IOPRIO_CLASS_IDLE: + return &bfqg->async_idle_bfqq; + default: + BUG(); + } +} + +static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, + struct bio *bio, bool is_sync, + struct bfq_io_cq *bic) +{ + const int ioprio = IOPRIO_PRIO_DATA(bic->ioprio); + const int ioprio_class = IOPRIO_PRIO_CLASS(bic->ioprio); + struct bfq_queue **async_bfqq = NULL; + struct bfq_queue *bfqq; + struct bfq_group *bfqg; + + rcu_read_lock(); + + bfqg = bfq_find_set_group(bfqd, bio_blkcg(bio)); + if (!bfqg) { + bfqq = &bfqd->oom_bfqq; + goto out; + } + + if (!is_sync) { + async_bfqq = bfq_async_queue_prio(bfqd, bfqg, ioprio_class, + ioprio); + bfqq = *async_bfqq; + if (bfqq) + goto out; + } + + bfqq = kmem_cache_alloc_node(bfq_pool, + GFP_NOWAIT | __GFP_ZERO | __GFP_NOWARN, + bfqd->queue->node); + + if (bfqq) { + bfq_init_bfqq(bfqd, bfqq, bic, current->pid, + is_sync); + bfq_init_entity(&bfqq->entity, bfqg); + bfq_log_bfqq(bfqd, bfqq, "allocated"); + } else { + bfqq = &bfqd->oom_bfqq; + bfq_log_bfqq(bfqd, bfqq, "using oom bfqq"); + goto out; + } + + /* + * Pin the queue now that it's allocated, scheduler exit will + * prune it. + */ + if (async_bfqq) { + bfqq->ref++; /* + * Extra group reference, w.r.t. sync + * queue. This extra reference is removed + * only if bfqq->bfqg disappears, to + * guarantee that this queue is not freed + * until its group goes away. + */ + bfq_log_bfqq(bfqd, bfqq, "get_queue, bfqq not in async: %p, %d", + bfqq, bfqq->ref); + *async_bfqq = bfqq; + } + +out: + bfqq->ref++; /* get a process reference to this queue */ + bfq_log_bfqq(bfqd, bfqq, "get_queue, at end: %p, %d", bfqq, bfqq->ref); + rcu_read_unlock(); + return bfqq; +} + +static void bfq_update_io_thinktime(struct bfq_data *bfqd, + struct bfq_io_cq *bic) +{ + struct bfq_ttime *ttime = &bic->ttime; + u64 elapsed = ktime_get_ns() - bic->ttime.last_end_request; + + elapsed = min_t(u64, elapsed, 2 * bfqd->bfq_slice_idle); + + ttime->ttime_samples = (7*bic->ttime.ttime_samples + 256) / 8; + ttime->ttime_total = div_u64(7*ttime->ttime_total + 256*elapsed, 8); + ttime->ttime_mean = div64_ul(ttime->ttime_total + 128, + ttime->ttime_samples); +} + +static void +bfq_update_io_seektime(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct request *rq) +{ + bfqq->seek_history <<= 1; + bfqq->seek_history |= + get_sdist(bfqq->last_request_pos, rq) > BFQQ_SEEK_THR && + (!blk_queue_nonrot(bfqd->queue) || + blk_rq_sectors(rq) < BFQQ_SECT_THR_NONROT); +} + +static void bfq_update_has_short_ttime(struct bfq_data *bfqd, + struct bfq_queue *bfqq, + struct bfq_io_cq *bic) +{ + bool has_short_ttime = true; + + /* + * No need to update has_short_ttime if bfqq is async or in + * idle io prio class, or if bfq_slice_idle is zero, because + * no device idling is performed for bfqq in this case. + */ + if (!bfq_bfqq_sync(bfqq) || bfq_class_idle(bfqq) || + bfqd->bfq_slice_idle == 0) + return; + + /* Idle window just restored, statistics are meaningless. */ + if (time_is_after_eq_jiffies(bfqq->split_time + + bfqd->bfq_wr_min_idle_time)) + return; + + /* Think time is infinite if no process is linked to + * bfqq. Otherwise check average think time to + * decide whether to mark as has_short_ttime + */ + if (atomic_read(&bic->icq.ioc->active_ref) == 0 || + (bfq_sample_valid(bic->ttime.ttime_samples) && + bic->ttime.ttime_mean > bfqd->bfq_slice_idle)) + has_short_ttime = false; + + bfq_log_bfqq(bfqd, bfqq, "update_has_short_ttime: has_short_ttime %d", + has_short_ttime); + + if (has_short_ttime) + bfq_mark_bfqq_has_short_ttime(bfqq); + else + bfq_clear_bfqq_has_short_ttime(bfqq); +} + +/* + * Called when a new fs request (rq) is added to bfqq. Check if there's + * something we should do about it. + */ +static void bfq_rq_enqueued(struct bfq_data *bfqd, struct bfq_queue *bfqq, + struct request *rq) +{ + struct bfq_io_cq *bic = RQ_BIC(rq); + + if (rq->cmd_flags & REQ_META) + bfqq->meta_pending++; + + bfq_update_io_thinktime(bfqd, bic); + bfq_update_has_short_ttime(bfqd, bfqq, bic); + bfq_update_io_seektime(bfqd, bfqq, rq); + + bfq_log_bfqq(bfqd, bfqq, + "rq_enqueued: has_short_ttime=%d (seeky %d)", + bfq_bfqq_has_short_ttime(bfqq), BFQQ_SEEKY(bfqq)); + + bfqq->last_request_pos = blk_rq_pos(rq) + blk_rq_sectors(rq); + + if (bfqq == bfqd->in_service_queue && bfq_bfqq_wait_request(bfqq)) { + bool small_req = bfqq->queued[rq_is_sync(rq)] == 1 && + blk_rq_sectors(rq) < 32; + bool budget_timeout = bfq_bfqq_budget_timeout(bfqq); + + /* + * There is just this request queued: if the request + * is small and the queue is not to be expired, then + * just exit. + * + * In this way, if the device is being idled to wait + * for a new request from the in-service queue, we + * avoid unplugging the device and committing the + * device to serve just a small request. On the + * contrary, we wait for the block layer to decide + * when to unplug the device: hopefully, new requests + * will be merged to this one quickly, then the device + * will be unplugged and larger requests will be + * dispatched. + */ + if (small_req && !budget_timeout) + return; + + /* + * A large enough request arrived, or the queue is to + * be expired: in both cases disk idling is to be + * stopped, so clear wait_request flag and reset + * timer. + */ + bfq_clear_bfqq_wait_request(bfqq); + hrtimer_try_to_cancel(&bfqd->idle_slice_timer); + bfqg_stats_update_idle_time(bfqq_group(bfqq)); + + /* + * The queue is not empty, because a new request just + * arrived. Hence we can safely expire the queue, in + * case of budget timeout, without risking that the + * timestamps of the queue are not updated correctly. + * See [1] for more details. + */ + if (budget_timeout) + bfq_bfqq_expire(bfqd, bfqq, false, + BFQ_BFQQ_BUDGET_TIMEOUT); + + /* + * Let the request rip immediately, or let a new queue be + * selected if bfqq has just been expired. + */ + __blk_run_queue(bfqd->queue); + } +} + +static void bfq_insert_request(struct request_queue *q, struct request *rq) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + struct bfq_queue *bfqq = RQ_BFQQ(rq), *new_bfqq; + + assert_spin_locked(bfqd->queue->queue_lock); + + /* + * An unplug may trigger a requeue of a request from the device + * driver: make sure we are in process context while trying to + * merge two bfq_queues. + */ + if (!in_interrupt()) { + new_bfqq = bfq_setup_cooperator(bfqd, bfqq, rq, true); + if (new_bfqq) { + if (bic_to_bfqq(RQ_BIC(rq), 1) != bfqq) + new_bfqq = bic_to_bfqq(RQ_BIC(rq), 1); + /* + * Release the request's reference to the old bfqq + * and make sure one is taken to the shared queue. + */ + new_bfqq->allocated[rq_data_dir(rq)]++; + bfqq->allocated[rq_data_dir(rq)]--; + new_bfqq->ref++; + bfq_clear_bfqq_just_created(bfqq); + if (bic_to_bfqq(RQ_BIC(rq), 1) == bfqq) + bfq_merge_bfqqs(bfqd, RQ_BIC(rq), + bfqq, new_bfqq); + /* + * rq is about to be enqueued into new_bfqq, + * release rq reference on bfqq + */ + bfq_put_queue(bfqq); + rq->elv.priv[1] = new_bfqq; + bfqq = new_bfqq; + } + } + + bfq_add_request(rq); + + rq->fifo_time = ktime_get_ns() + bfqd->bfq_fifo_expire[rq_is_sync(rq)]; + list_add_tail(&rq->queuelist, &bfqq->fifo); + + bfq_rq_enqueued(bfqd, bfqq, rq); +} + +static void bfq_update_hw_tag(struct bfq_data *bfqd) +{ + bfqd->max_rq_in_driver = max_t(int, bfqd->max_rq_in_driver, + bfqd->rq_in_driver); + + if (bfqd->hw_tag == 1) + return; + + /* + * This sample is valid if the number of outstanding requests + * is large enough to allow a queueing behavior. Note that the + * sum is not exact, as it's not taking into account deactivated + * requests. + */ + if (bfqd->rq_in_driver + bfqd->queued < BFQ_HW_QUEUE_THRESHOLD) + return; + + if (bfqd->hw_tag_samples++ < BFQ_HW_QUEUE_SAMPLES) + return; + + bfqd->hw_tag = bfqd->max_rq_in_driver > BFQ_HW_QUEUE_THRESHOLD; + bfqd->max_rq_in_driver = 0; + bfqd->hw_tag_samples = 0; +} + +static void bfq_completed_request(struct request_queue *q, struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + struct bfq_data *bfqd = bfqq->bfqd; + u64 now_ns; + u32 delta_us; + + bfq_log_bfqq(bfqd, bfqq, "completed one req with %u sects left", + blk_rq_sectors(rq)); + + assert_spin_locked(bfqd->queue->queue_lock); + bfq_update_hw_tag(bfqd); + + BUG_ON(!bfqd->rq_in_driver); + BUG_ON(!bfqq->dispatched); + bfqd->rq_in_driver--; + bfqq->dispatched--; + bfqg_stats_update_completion(bfqq_group(bfqq), + rq_start_time_ns(rq), + rq_io_start_time_ns(rq), + rq->cmd_flags); + + if (!bfqq->dispatched && !bfq_bfqq_busy(bfqq)) { + BUG_ON(!RB_EMPTY_ROOT(&bfqq->sort_list)); + /* + * Set budget_timeout (which we overload to store the + * time at which the queue remains with no backlog and + * no outstanding request; used by the weight-raising + * mechanism). + */ + bfqq->budget_timeout = jiffies; + + bfq_weights_tree_remove(bfqd, &bfqq->entity, + &bfqd->queue_weights_tree); + } + + now_ns = ktime_get_ns(); + + RQ_BIC(rq)->ttime.last_end_request = now_ns; + + /* + * Using us instead of ns, to get a reasonable precision in + * computing rate in next check. + */ + delta_us = div_u64(now_ns - bfqd->last_completion, NSEC_PER_USEC); + + bfq_log(bfqd, "rq_completed: delta %uus/%luus max_size %u rate %llu/%llu", + delta_us, BFQ_MIN_TT/NSEC_PER_USEC, bfqd->last_rq_max_size, + (USEC_PER_SEC* + (u64)((bfqd->last_rq_max_size<>BFQ_RATE_SHIFT, + (USEC_PER_SEC*(u64)(1UL<<(BFQ_RATE_SHIFT-10)))>>BFQ_RATE_SHIFT); + + /* + * If the request took rather long to complete, and, according + * to the maximum request size recorded, this completion latency + * implies that the request was certainly served at a very low + * rate (less than 1M sectors/sec), then the whole observation + * interval that lasts up to this time instant cannot be a + * valid time interval for computing a new peak rate. Invoke + * bfq_update_rate_reset to have the following three steps + * taken: + * - close the observation interval at the last (previous) + * request dispatch or completion + * - compute rate, if possible, for that observation interval + * - reset to zero samples, which will trigger a proper + * re-initialization of the observation interval on next + * dispatch + */ + if (delta_us > BFQ_MIN_TT/NSEC_PER_USEC && + (bfqd->last_rq_max_size<last_completion = now_ns; + + /* + * If we are waiting to discover whether the request pattern + * of the task associated with the queue is actually + * isochronous, and both requisites for this condition to hold + * are now satisfied, then compute soft_rt_next_start (see the + * comments on the function bfq_bfqq_softrt_next_start()). We + * schedule this delayed check when bfqq expires, if it still + * has in-flight requests. + */ + if (bfq_bfqq_softrt_update(bfqq) && bfqq->dispatched == 0 && + RB_EMPTY_ROOT(&bfqq->sort_list)) + bfqq->soft_rt_next_start = + bfq_bfqq_softrt_next_start(bfqd, bfqq); + + /* + * If this is the in-service queue, check if it needs to be expired, + * or if we want to idle in case it has no pending requests. + */ + if (bfqd->in_service_queue == bfqq) { + if (bfqq->dispatched == 0 && bfq_bfqq_must_idle(bfqq)) { + bfq_arm_slice_timer(bfqd); + goto out; + } else if (bfq_may_expire_for_budg_timeout(bfqq)) + bfq_bfqq_expire(bfqd, bfqq, false, + BFQ_BFQQ_BUDGET_TIMEOUT); + else if (RB_EMPTY_ROOT(&bfqq->sort_list) && + (bfqq->dispatched == 0 || + !bfq_bfqq_may_idle(bfqq))) + bfq_bfqq_expire(bfqd, bfqq, false, + BFQ_BFQQ_NO_MORE_REQUESTS); + } + + if (!bfqd->rq_in_driver) + bfq_schedule_dispatch(bfqd); + +out: + return; +} + +static int __bfq_may_queue(struct bfq_queue *bfqq) +{ + if (bfq_bfqq_wait_request(bfqq) && bfq_bfqq_must_alloc(bfqq)) { + bfq_clear_bfqq_must_alloc(bfqq); + return ELV_MQUEUE_MUST; + } + + return ELV_MQUEUE_MAY; +} + +static int bfq_may_queue(struct request_queue *q, unsigned int op) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + struct task_struct *tsk = current; + struct bfq_io_cq *bic; + struct bfq_queue *bfqq; + + /* + * Don't force setup of a queue from here, as a call to may_queue + * does not necessarily imply that a request actually will be + * queued. So just lookup a possibly existing queue, or return + * 'may queue' if that fails. + */ + bic = bfq_bic_lookup(bfqd, tsk->io_context); + if (!bic) + return ELV_MQUEUE_MAY; + + bfqq = bic_to_bfqq(bic, op_is_sync(op)); + if (bfqq) + return __bfq_may_queue(bfqq); + + return ELV_MQUEUE_MAY; +} + +/* + * Queue lock held here. + */ +static void bfq_put_request(struct request *rq) +{ + struct bfq_queue *bfqq = RQ_BFQQ(rq); + + if (bfqq) { + const int rw = rq_data_dir(rq); + + BUG_ON(!bfqq->allocated[rw]); + bfqq->allocated[rw]--; + + rq->elv.priv[0] = NULL; + rq->elv.priv[1] = NULL; + + bfq_log_bfqq(bfqq->bfqd, bfqq, "put_request %p, %d", + bfqq, bfqq->ref); + bfq_put_queue(bfqq); + } +} + +/* + * Returns NULL if a new bfqq should be allocated, or the old bfqq if this + * was the last process referring to that bfqq. + */ +static struct bfq_queue * +bfq_split_bfqq(struct bfq_io_cq *bic, struct bfq_queue *bfqq) +{ + bfq_log_bfqq(bfqq->bfqd, bfqq, "splitting queue"); + + put_io_context(bic->icq.ioc); + + if (bfqq_process_refs(bfqq) == 1) { + bfqq->pid = current->pid; + bfq_clear_bfqq_coop(bfqq); + bfq_clear_bfqq_split_coop(bfqq); + return bfqq; + } + + bic_set_bfqq(bic, NULL, 1); + + bfq_put_cooperator(bfqq); + + bfq_put_queue(bfqq); + return NULL; +} + +/* + * Allocate bfq data structures associated with this request. + */ +static int bfq_set_request(struct request_queue *q, struct request *rq, + struct bio *bio, gfp_t gfp_mask) +{ + struct bfq_data *bfqd = q->elevator->elevator_data; + struct bfq_io_cq *bic = icq_to_bic(rq->elv.icq); + const int rw = rq_data_dir(rq); + const int is_sync = rq_is_sync(rq); + struct bfq_queue *bfqq; + unsigned long flags; + bool bfqq_already_existing = false, split = false; + + spin_lock_irqsave(q->queue_lock, flags); + + if (!bic) + goto queue_fail; + + bfq_check_ioprio_change(bic, bio); + + bfq_bic_update_cgroup(bic, bio); + +new_queue: + bfqq = bic_to_bfqq(bic, is_sync); + if (!bfqq || bfqq == &bfqd->oom_bfqq) { + if (bfqq) + bfq_put_queue(bfqq); + bfqq = bfq_get_queue(bfqd, bio, is_sync, bic); + BUG_ON(!hlist_unhashed(&bfqq->burst_list_node)); + + bic_set_bfqq(bic, bfqq, is_sync); + if (split && is_sync) { + bfq_log_bfqq(bfqd, bfqq, + "set_request: was_in_list %d " + "was_in_large_burst %d " + "large burst in progress %d", + bic->was_in_burst_list, + bic->saved_in_large_burst, + bfqd->large_burst); + + if ((bic->was_in_burst_list && bfqd->large_burst) || + bic->saved_in_large_burst) { + bfq_log_bfqq(bfqd, bfqq, + "set_request: marking in " + "large burst"); + bfq_mark_bfqq_in_large_burst(bfqq); + } else { + bfq_log_bfqq(bfqd, bfqq, + "set_request: clearing in " + "large burst"); + bfq_clear_bfqq_in_large_burst(bfqq); + if (bic->was_in_burst_list) + hlist_add_head(&bfqq->burst_list_node, + &bfqd->burst_list); + } + bfqq->split_time = jiffies; + } + } else { + /* If the queue was seeky for too long, break it apart. */ + if (bfq_bfqq_coop(bfqq) && bfq_bfqq_split_coop(bfqq)) { + bfq_log_bfqq(bfqd, bfqq, "breaking apart bfqq"); + + /* Update bic before losing reference to bfqq */ + if (bfq_bfqq_in_large_burst(bfqq)) + bic->saved_in_large_burst = true; + + bfqq = bfq_split_bfqq(bic, bfqq); + split = true; + if (!bfqq) + goto new_queue; + else + bfqq_already_existing = true; + } + } + + bfqq->allocated[rw]++; + bfqq->ref++; + bfq_log_bfqq(bfqd, bfqq, "set_request: bfqq %p, %d", bfqq, bfqq->ref); + + rq->elv.priv[0] = bic; + rq->elv.priv[1] = bfqq; + + /* + * If a bfq_queue has only one process reference, it is owned + * by only one bfq_io_cq: we can set the bic field of the + * bfq_queue to the address of that structure. Also, if the + * queue has just been split, mark a flag so that the + * information is available to the other scheduler hooks. + */ + if (likely(bfqq != &bfqd->oom_bfqq) && bfqq_process_refs(bfqq) == 1) { + bfqq->bic = bic; + if (split) { + /* + * If the queue has just been split from a shared + * queue, restore the idle window and the possible + * weight raising period. + */ + bfq_bfqq_resume_state(bfqq, bfqd, bic, + bfqq_already_existing); + } + } + + if (unlikely(bfq_bfqq_just_created(bfqq))) + bfq_handle_burst(bfqd, bfqq); + + spin_unlock_irqrestore(q->queue_lock, flags); + + return 0; + +queue_fail: + bfq_schedule_dispatch(bfqd); + spin_unlock_irqrestore(q->queue_lock, flags); + + return 1; +} + +static void bfq_kick_queue(struct work_struct *work) +{ + struct bfq_data *bfqd = + container_of(work, struct bfq_data, unplug_work); + struct request_queue *q = bfqd->queue; + + spin_lock_irq(q->queue_lock); + __blk_run_queue(q); + spin_unlock_irq(q->queue_lock); +} + +/* + * Handler of the expiration of the timer running if the in-service queue + * is idling inside its time slice. + */ +static enum hrtimer_restart bfq_idle_slice_timer(struct hrtimer *timer) +{ + struct bfq_data *bfqd = container_of(timer, struct bfq_data, + idle_slice_timer); + struct bfq_queue *bfqq; + unsigned long flags; + enum bfqq_expiration reason; + + spin_lock_irqsave(bfqd->queue->queue_lock, flags); + + bfqq = bfqd->in_service_queue; + /* + * Theoretical race here: the in-service queue can be NULL or + * different from the queue that was idling if the timer handler + * spins on the queue_lock and a new request arrives for the + * current queue and there is a full dispatch cycle that changes + * the in-service queue. This can hardly happen, but in the worst + * case we just expire a queue too early. + */ + if (bfqq) { + bfq_log_bfqq(bfqd, bfqq, "slice_timer expired"); + bfq_clear_bfqq_wait_request(bfqq); + + if (bfq_bfqq_budget_timeout(bfqq)) + /* + * Also here the queue can be safely expired + * for budget timeout without wasting + * guarantees + */ + reason = BFQ_BFQQ_BUDGET_TIMEOUT; + else if (bfqq->queued[0] == 0 && bfqq->queued[1] == 0) + /* + * The queue may not be empty upon timer expiration, + * because we may not disable the timer when the + * first request of the in-service queue arrives + * during disk idling. + */ + reason = BFQ_BFQQ_TOO_IDLE; + else + goto schedule_dispatch; + + bfq_bfqq_expire(bfqd, bfqq, true, reason); + } + +schedule_dispatch: + bfq_schedule_dispatch(bfqd); + + spin_unlock_irqrestore(bfqd->queue->queue_lock, flags); + return HRTIMER_NORESTART; +} + +static void bfq_shutdown_timer_wq(struct bfq_data *bfqd) +{ + hrtimer_cancel(&bfqd->idle_slice_timer); + cancel_work_sync(&bfqd->unplug_work); +} + +static void __bfq_put_async_bfqq(struct bfq_data *bfqd, + struct bfq_queue **bfqq_ptr) +{ + struct bfq_group *root_group = bfqd->root_group; + struct bfq_queue *bfqq = *bfqq_ptr; + + bfq_log(bfqd, "put_async_bfqq: %p", bfqq); + if (bfqq) { + bfq_bfqq_move(bfqd, bfqq, root_group); + bfq_log_bfqq(bfqd, bfqq, "put_async_bfqq: putting %p, %d", + bfqq, bfqq->ref); + bfq_put_queue(bfqq); + *bfqq_ptr = NULL; + } +} + +/* + * Release all the bfqg references to its async queues. If we are + * deallocating the group these queues may still contain requests, so + * we reparent them to the root cgroup (i.e., the only one that will + * exist for sure until all the requests on a device are gone). + */ +static void bfq_put_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg) +{ + int i, j; + + for (i = 0; i < 2; i++) + for (j = 0; j < IOPRIO_BE_NR; j++) + __bfq_put_async_bfqq(bfqd, &bfqg->async_bfqq[i][j]); + + __bfq_put_async_bfqq(bfqd, &bfqg->async_idle_bfqq); +} + +static void bfq_exit_queue(struct elevator_queue *e) +{ + struct bfq_data *bfqd = e->elevator_data; + struct request_queue *q = bfqd->queue; + struct bfq_queue *bfqq, *n; + + bfq_shutdown_timer_wq(bfqd); + + spin_lock_irq(q->queue_lock); + + BUG_ON(bfqd->in_service_queue); + list_for_each_entry_safe(bfqq, n, &bfqd->idle_list, bfqq_list) + bfq_deactivate_bfqq(bfqd, bfqq, false, false); + + spin_unlock_irq(q->queue_lock); + + bfq_shutdown_timer_wq(bfqd); + + BUG_ON(hrtimer_active(&bfqd->idle_slice_timer)); + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + blkcg_deactivate_policy(q, &blkcg_policy_bfq); +#else + bfq_put_async_queues(bfqd, bfqd->root_group); + kfree(bfqd->root_group); +#endif + + kfree(bfqd); +} + +static void bfq_init_root_group(struct bfq_group *root_group, + struct bfq_data *bfqd) +{ + int i; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + root_group->entity.parent = NULL; + root_group->my_entity = NULL; + root_group->bfqd = bfqd; +#endif + root_group->rq_pos_tree = RB_ROOT; + for (i = 0; i < BFQ_IOPRIO_CLASSES; i++) + root_group->sched_data.service_tree[i] = BFQ_SERVICE_TREE_INIT; + root_group->sched_data.bfq_class_idle_last_service = jiffies; +} + +static int bfq_init_queue(struct request_queue *q, struct elevator_type *e) +{ + struct bfq_data *bfqd; + struct elevator_queue *eq; + + eq = elevator_alloc(q, e); + if (!eq) + return -ENOMEM; + + bfqd = kzalloc_node(sizeof(*bfqd), GFP_KERNEL, q->node); + if (!bfqd) { + kobject_put(&eq->kobj); + return -ENOMEM; + } + eq->elevator_data = bfqd; + + /* + * Our fallback bfqq if bfq_find_alloc_queue() runs into OOM issues. + * Grab a permanent reference to it, so that the normal code flow + * will not attempt to free it. + */ + bfq_init_bfqq(bfqd, &bfqd->oom_bfqq, NULL, 1, 0); + bfqd->oom_bfqq.ref++; + bfqd->oom_bfqq.new_ioprio = BFQ_DEFAULT_QUEUE_IOPRIO; + bfqd->oom_bfqq.new_ioprio_class = IOPRIO_CLASS_BE; + bfqd->oom_bfqq.entity.new_weight = + bfq_ioprio_to_weight(bfqd->oom_bfqq.new_ioprio); + + /* oom_bfqq does not participate to bursts */ + bfq_clear_bfqq_just_created(&bfqd->oom_bfqq); + /* + * Trigger weight initialization, according to ioprio, at the + * oom_bfqq's first activation. The oom_bfqq's ioprio and ioprio + * class won't be changed any more. + */ + bfqd->oom_bfqq.entity.prio_changed = 1; + + bfqd->queue = q; + + spin_lock_irq(q->queue_lock); + q->elevator = eq; + spin_unlock_irq(q->queue_lock); + + bfqd->root_group = bfq_create_group_hierarchy(bfqd, q->node); + if (!bfqd->root_group) + goto out_free; + bfq_init_root_group(bfqd->root_group, bfqd); + bfq_init_entity(&bfqd->oom_bfqq.entity, bfqd->root_group); + + hrtimer_init(&bfqd->idle_slice_timer, CLOCK_MONOTONIC, + HRTIMER_MODE_REL); + bfqd->idle_slice_timer.function = bfq_idle_slice_timer; + + bfqd->queue_weights_tree = RB_ROOT; + bfqd->group_weights_tree = RB_ROOT; + + INIT_WORK(&bfqd->unplug_work, bfq_kick_queue); + + INIT_LIST_HEAD(&bfqd->active_list); + INIT_LIST_HEAD(&bfqd->idle_list); + INIT_HLIST_HEAD(&bfqd->burst_list); + + bfqd->hw_tag = -1; + + bfqd->bfq_max_budget = bfq_default_max_budget; + + bfqd->bfq_fifo_expire[0] = bfq_fifo_expire[0]; + bfqd->bfq_fifo_expire[1] = bfq_fifo_expire[1]; + bfqd->bfq_back_max = bfq_back_max; + bfqd->bfq_back_penalty = bfq_back_penalty; + bfqd->bfq_slice_idle = bfq_slice_idle; + bfqd->bfq_timeout = bfq_timeout; + + bfqd->bfq_requests_within_timer = 120; + + bfqd->bfq_large_burst_thresh = 8; + bfqd->bfq_burst_interval = msecs_to_jiffies(180); + + bfqd->low_latency = true; + + /* + * Trade-off between responsiveness and fairness. + */ + bfqd->bfq_wr_coeff = 30; + bfqd->bfq_wr_rt_max_time = msecs_to_jiffies(300); + bfqd->bfq_wr_max_time = 0; + bfqd->bfq_wr_min_idle_time = msecs_to_jiffies(2000); + bfqd->bfq_wr_min_inter_arr_async = msecs_to_jiffies(500); + bfqd->bfq_wr_max_softrt_rate = 7000; /* + * Approximate rate required + * to playback or record a + * high-definition compressed + * video. + */ + bfqd->wr_busy_queues = 0; + + /* + * Begin by assuming, optimistically, that the device is a + * high-speed one, and that its peak rate is equal to 2/3 of + * the highest reference rate. + */ + bfqd->RT_prod = R_fast[blk_queue_nonrot(bfqd->queue)] * + T_fast[blk_queue_nonrot(bfqd->queue)]; + bfqd->peak_rate = R_fast[blk_queue_nonrot(bfqd->queue)] * 2 / 3; + bfqd->device_speed = BFQ_BFQD_FAST; + + return 0; + +out_free: + kfree(bfqd); + kobject_put(&eq->kobj); + return -ENOMEM; +} + +static void bfq_slab_kill(void) +{ + kmem_cache_destroy(bfq_pool); +} + +static int __init bfq_slab_setup(void) +{ + bfq_pool = KMEM_CACHE(bfq_queue, 0); + if (!bfq_pool) + return -ENOMEM; + return 0; +} + +static ssize_t bfq_var_show(unsigned int var, char *page) +{ + return sprintf(page, "%u\n", var); +} + +static ssize_t bfq_var_store(unsigned long *var, const char *page, + size_t count) +{ + unsigned long new_val; + int ret = kstrtoul(page, 10, &new_val); + + if (ret == 0) + *var = new_val; + + return count; +} + +static ssize_t bfq_wr_max_time_show(struct elevator_queue *e, char *page) +{ + struct bfq_data *bfqd = e->elevator_data; + + return sprintf(page, "%d\n", bfqd->bfq_wr_max_time > 0 ? + jiffies_to_msecs(bfqd->bfq_wr_max_time) : + jiffies_to_msecs(bfq_wr_duration(bfqd))); +} + +static ssize_t bfq_weights_show(struct elevator_queue *e, char *page) +{ + struct bfq_queue *bfqq; + struct bfq_data *bfqd = e->elevator_data; + ssize_t num_char = 0; + + num_char += sprintf(page + num_char, "Tot reqs queued %d\n\n", + bfqd->queued); + + spin_lock_irq(bfqd->queue->queue_lock); + + num_char += sprintf(page + num_char, "Active:\n"); + list_for_each_entry(bfqq, &bfqd->active_list, bfqq_list) { + num_char += sprintf(page + num_char, + "pid%d: weight %hu, nr_queued %d %d, ", + bfqq->pid, + bfqq->entity.weight, + bfqq->queued[0], + bfqq->queued[1]); + num_char += sprintf(page + num_char, + "dur %d/%u\n", + jiffies_to_msecs( + jiffies - + bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } + + num_char += sprintf(page + num_char, "Idle:\n"); + list_for_each_entry(bfqq, &bfqd->idle_list, bfqq_list) { + num_char += sprintf(page + num_char, + "pid%d: weight %hu, dur %d/%u\n", + bfqq->pid, + bfqq->entity.weight, + jiffies_to_msecs(jiffies - + bfqq->last_wr_start_finish), + jiffies_to_msecs(bfqq->wr_cur_max_time)); + } + + spin_unlock_irq(bfqd->queue->queue_lock); + + return num_char; +} + +#define SHOW_FUNCTION(__FUNC, __VAR, __CONV) \ +static ssize_t __FUNC(struct elevator_queue *e, char *page) \ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + u64 __data = __VAR; \ + if (__CONV == 1) \ + __data = jiffies_to_msecs(__data); \ + else if (__CONV == 2) \ + __data = div_u64(__data, NSEC_PER_MSEC); \ + return bfq_var_show(__data, (page)); \ +} +SHOW_FUNCTION(bfq_fifo_expire_sync_show, bfqd->bfq_fifo_expire[1], 2); +SHOW_FUNCTION(bfq_fifo_expire_async_show, bfqd->bfq_fifo_expire[0], 2); +SHOW_FUNCTION(bfq_back_seek_max_show, bfqd->bfq_back_max, 0); +SHOW_FUNCTION(bfq_back_seek_penalty_show, bfqd->bfq_back_penalty, 0); +SHOW_FUNCTION(bfq_slice_idle_show, bfqd->bfq_slice_idle, 2); +SHOW_FUNCTION(bfq_max_budget_show, bfqd->bfq_user_max_budget, 0); +SHOW_FUNCTION(bfq_timeout_sync_show, bfqd->bfq_timeout, 1); +SHOW_FUNCTION(bfq_strict_guarantees_show, bfqd->strict_guarantees, 0); +SHOW_FUNCTION(bfq_low_latency_show, bfqd->low_latency, 0); +SHOW_FUNCTION(bfq_wr_coeff_show, bfqd->bfq_wr_coeff, 0); +SHOW_FUNCTION(bfq_wr_rt_max_time_show, bfqd->bfq_wr_rt_max_time, 1); +SHOW_FUNCTION(bfq_wr_min_idle_time_show, bfqd->bfq_wr_min_idle_time, 1); +SHOW_FUNCTION(bfq_wr_min_inter_arr_async_show, bfqd->bfq_wr_min_inter_arr_async, + 1); +SHOW_FUNCTION(bfq_wr_max_softrt_rate_show, bfqd->bfq_wr_max_softrt_rate, 0); +#undef SHOW_FUNCTION + +#define USEC_SHOW_FUNCTION(__FUNC, __VAR) \ +static ssize_t __FUNC(struct elevator_queue *e, char *page) \ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + u64 __data = __VAR; \ + __data = div_u64(__data, NSEC_PER_USEC); \ + return bfq_var_show(__data, (page)); \ +} +USEC_SHOW_FUNCTION(bfq_slice_idle_us_show, bfqd->bfq_slice_idle); +#undef USEC_SHOW_FUNCTION + +#define STORE_FUNCTION(__FUNC, __PTR, MIN, MAX, __CONV) \ +static ssize_t \ +__FUNC(struct elevator_queue *e, const char *page, size_t count) \ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + unsigned long uninitialized_var(__data); \ + int ret = bfq_var_store(&__data, (page), count); \ + if (__data < (MIN)) \ + __data = (MIN); \ + else if (__data > (MAX)) \ + __data = (MAX); \ + if (__CONV == 1) \ + *(__PTR) = msecs_to_jiffies(__data); \ + else if (__CONV == 2) \ + *(__PTR) = (u64)__data * NSEC_PER_MSEC; \ + else \ + *(__PTR) = __data; \ + return ret; \ +} +STORE_FUNCTION(bfq_fifo_expire_sync_store, &bfqd->bfq_fifo_expire[1], 1, + INT_MAX, 2); +STORE_FUNCTION(bfq_fifo_expire_async_store, &bfqd->bfq_fifo_expire[0], 1, + INT_MAX, 2); +STORE_FUNCTION(bfq_back_seek_max_store, &bfqd->bfq_back_max, 0, INT_MAX, 0); +STORE_FUNCTION(bfq_back_seek_penalty_store, &bfqd->bfq_back_penalty, 1, + INT_MAX, 0); +STORE_FUNCTION(bfq_slice_idle_store, &bfqd->bfq_slice_idle, 0, INT_MAX, 2); +STORE_FUNCTION(bfq_wr_coeff_store, &bfqd->bfq_wr_coeff, 1, INT_MAX, 0); +STORE_FUNCTION(bfq_wr_max_time_store, &bfqd->bfq_wr_max_time, 0, INT_MAX, 1); +STORE_FUNCTION(bfq_wr_rt_max_time_store, &bfqd->bfq_wr_rt_max_time, 0, INT_MAX, + 1); +STORE_FUNCTION(bfq_wr_min_idle_time_store, &bfqd->bfq_wr_min_idle_time, 0, + INT_MAX, 1); +STORE_FUNCTION(bfq_wr_min_inter_arr_async_store, + &bfqd->bfq_wr_min_inter_arr_async, 0, INT_MAX, 1); +STORE_FUNCTION(bfq_wr_max_softrt_rate_store, &bfqd->bfq_wr_max_softrt_rate, 0, + INT_MAX, 0); +#undef STORE_FUNCTION + +#define USEC_STORE_FUNCTION(__FUNC, __PTR, MIN, MAX) \ +static ssize_t __FUNC(struct elevator_queue *e, const char *page, size_t count)\ +{ \ + struct bfq_data *bfqd = e->elevator_data; \ + unsigned long uninitialized_var(__data); \ + int ret = bfq_var_store(&__data, (page), count); \ + if (__data < (MIN)) \ + __data = (MIN); \ + else if (__data > (MAX)) \ + __data = (MAX); \ + *(__PTR) = (u64)__data * NSEC_PER_USEC; \ + return ret; \ +} +USEC_STORE_FUNCTION(bfq_slice_idle_us_store, &bfqd->bfq_slice_idle, 0, + UINT_MAX); +#undef USEC_STORE_FUNCTION + +/* do nothing for the moment */ +static ssize_t bfq_weights_store(struct elevator_queue *e, + const char *page, size_t count) +{ + return count; +} + +static ssize_t bfq_max_budget_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data == 0) + bfqd->bfq_max_budget = bfq_calc_max_budget(bfqd); + else { + if (__data > INT_MAX) + __data = INT_MAX; + bfqd->bfq_max_budget = __data; + } + + bfqd->bfq_user_max_budget = __data; + + return ret; +} + +/* + * Leaving this name to preserve name compatibility with cfq + * parameters, but this timeout is used for both sync and async. + */ +static ssize_t bfq_timeout_sync_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data < 1) + __data = 1; + else if (__data > INT_MAX) + __data = INT_MAX; + + bfqd->bfq_timeout = msecs_to_jiffies(__data); + if (bfqd->bfq_user_max_budget == 0) + bfqd->bfq_max_budget = bfq_calc_max_budget(bfqd); + + return ret; +} + +static ssize_t bfq_strict_guarantees_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data > 1) + __data = 1; + if (!bfqd->strict_guarantees && __data == 1 + && bfqd->bfq_slice_idle < 8 * NSEC_PER_MSEC) + bfqd->bfq_slice_idle = 8 * NSEC_PER_MSEC; + + bfqd->strict_guarantees = __data; + + return ret; +} + +static ssize_t bfq_low_latency_store(struct elevator_queue *e, + const char *page, size_t count) +{ + struct bfq_data *bfqd = e->elevator_data; + unsigned long uninitialized_var(__data); + int ret = bfq_var_store(&__data, (page), count); + + if (__data > 1) + __data = 1; + if (__data == 0 && bfqd->low_latency != 0) + bfq_end_wr(bfqd); + bfqd->low_latency = __data; + + return ret; +} + +#define BFQ_ATTR(name) \ + __ATTR(name, S_IRUGO|S_IWUSR, bfq_##name##_show, bfq_##name##_store) + +static struct elv_fs_entry bfq_attrs[] = { + BFQ_ATTR(fifo_expire_sync), + BFQ_ATTR(fifo_expire_async), + BFQ_ATTR(back_seek_max), + BFQ_ATTR(back_seek_penalty), + BFQ_ATTR(slice_idle), + BFQ_ATTR(slice_idle_us), + BFQ_ATTR(max_budget), + BFQ_ATTR(timeout_sync), + BFQ_ATTR(strict_guarantees), + BFQ_ATTR(low_latency), + BFQ_ATTR(wr_coeff), + BFQ_ATTR(wr_max_time), + BFQ_ATTR(wr_rt_max_time), + BFQ_ATTR(wr_min_idle_time), + BFQ_ATTR(wr_min_inter_arr_async), + BFQ_ATTR(wr_max_softrt_rate), + BFQ_ATTR(weights), + __ATTR_NULL +}; + +static struct elevator_type iosched_bfq = { + .ops.sq = { + .elevator_merge_fn = bfq_merge, + .elevator_merged_fn = bfq_merged_request, + .elevator_merge_req_fn = bfq_merged_requests, +#ifdef BFQ_GROUP_IOSCHED_ENABLED + .elevator_bio_merged_fn = bfq_bio_merged, +#endif + .elevator_allow_bio_merge_fn = bfq_allow_bio_merge, + .elevator_allow_rq_merge_fn = bfq_allow_rq_merge, + .elevator_dispatch_fn = bfq_dispatch_requests, + .elevator_add_req_fn = bfq_insert_request, + .elevator_activate_req_fn = bfq_activate_request, + .elevator_deactivate_req_fn = bfq_deactivate_request, + .elevator_completed_req_fn = bfq_completed_request, + .elevator_former_req_fn = elv_rb_former_request, + .elevator_latter_req_fn = elv_rb_latter_request, + .elevator_init_icq_fn = bfq_init_icq, + .elevator_exit_icq_fn = bfq_exit_icq, + .elevator_set_req_fn = bfq_set_request, + .elevator_put_req_fn = bfq_put_request, + .elevator_may_queue_fn = bfq_may_queue, + .elevator_init_fn = bfq_init_queue, + .elevator_exit_fn = bfq_exit_queue, + }, + .icq_size = sizeof(struct bfq_io_cq), + .icq_align = __alignof__(struct bfq_io_cq), + .elevator_attrs = bfq_attrs, + .elevator_name = "bfq-sq", + .elevator_owner = THIS_MODULE, +}; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static struct blkcg_policy blkcg_policy_bfq = { + .dfl_cftypes = bfq_blkg_files, + .legacy_cftypes = bfq_blkcg_legacy_files, + + .cpd_alloc_fn = bfq_cpd_alloc, + .cpd_init_fn = bfq_cpd_init, + .cpd_bind_fn = bfq_cpd_init, + .cpd_free_fn = bfq_cpd_free, + + .pd_alloc_fn = bfq_pd_alloc, + .pd_init_fn = bfq_pd_init, + .pd_offline_fn = bfq_pd_offline, + .pd_free_fn = bfq_pd_free, + .pd_reset_stats_fn = bfq_pd_reset_stats, +}; +#endif + +static int __init bfq_init(void) +{ + int ret; + char msg[60] = "BFQ I/O-scheduler: v8r12"; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + ret = blkcg_policy_register(&blkcg_policy_bfq); + if (ret) + return ret; +#endif + + ret = -ENOMEM; + if (bfq_slab_setup()) + goto err_pol_unreg; + + /* + * Times to load large popular applications for the typical + * systems installed on the reference devices (see the + * comments before the definitions of the next two + * arrays). Actually, we use slightly slower values, as the + * estimated peak rate tends to be smaller than the actual + * peak rate. The reason for this last fact is that estimates + * are computed over much shorter time intervals than the long + * intervals typically used for benchmarking. Why? First, to + * adapt more quickly to variations. Second, because an I/O + * scheduler cannot rely on a peak-rate-evaluation workload to + * be run for a long time. + */ + T_slow[0] = msecs_to_jiffies(3500); /* actually 4 sec */ + T_slow[1] = msecs_to_jiffies(6000); /* actually 6.5 sec */ + T_fast[0] = msecs_to_jiffies(7000); /* actually 8 sec */ + T_fast[1] = msecs_to_jiffies(2500); /* actually 3 sec */ + + /* + * Thresholds that determine the switch between speed classes + * (see the comments before the definition of the array + * device_speed_thresh). These thresholds are biased towards + * transitions to the fast class. This is safer than the + * opposite bias. In fact, a wrong transition to the slow + * class results in short weight-raising periods, because the + * speed of the device then tends to be higher that the + * reference peak rate. On the opposite end, a wrong + * transition to the fast class tends to increase + * weight-raising periods, because of the opposite reason. + */ + device_speed_thresh[0] = (4 * R_slow[0]) / 3; + device_speed_thresh[1] = (4 * R_slow[1]) / 3; + + ret = elv_register(&iosched_bfq); + if (ret) + goto err_pol_unreg; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + strcat(msg, " (with cgroups support)"); +#endif + pr_info("%s", msg); + + return 0; + +err_pol_unreg: +#ifdef BFQ_GROUP_IOSCHED_ENABLED + blkcg_policy_unregister(&blkcg_policy_bfq); +#endif + return ret; +} + +static void __exit bfq_exit(void) +{ + elv_unregister(&iosched_bfq); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + blkcg_policy_unregister(&blkcg_policy_bfq); +#endif + bfq_slab_kill(); +} + +module_init(bfq_init); +module_exit(bfq_exit); + +MODULE_AUTHOR("Arianna Avanzini, Fabio Checconi, Paolo Valente"); +MODULE_LICENSE("GPL"); diff --git b/block/bfq.h b/block/bfq.h new file mode 100644 index 0000000..15d326f --- /dev/null +++ b/block/bfq.h @@ -0,0 +1,989 @@ +/* + * BFQ v8r12 for 4.11.0: data structures and common functions prototypes. + * + * Based on ideas and code from CFQ: + * Copyright (C) 2003 Jens Axboe + * + * Copyright (C) 2008 Fabio Checconi + * Paolo Valente + * + * Copyright (C) 2015 Paolo Valente + * + * Copyright (C) 2017 Paolo Valente + */ + +#ifndef _BFQ_H +#define _BFQ_H + +#include +#include + +/* + * Define an alternative macro to compile cgroups support. This is one + * of the steps needed to let bfq-mq share the files bfq-sched.c and + * bfq-cgroup.c with bfq-sq. For bfq-mq, the macro + * BFQ_GROUP_IOSCHED_ENABLED will be defined as a function of whether + * the configuration option CONFIG_BFQ_MQ_GROUP_IOSCHED, and not + * CONFIG_BFQ_GROUP_IOSCHED, is defined. + */ +#ifdef CONFIG_BFQ_SQ_GROUP_IOSCHED +#define BFQ_GROUP_IOSCHED_ENABLED +#endif + +#define BFQ_IOPRIO_CLASSES 3 +#define BFQ_CL_IDLE_TIMEOUT (HZ/5) + +#define BFQ_MIN_WEIGHT 1 +#define BFQ_MAX_WEIGHT 1000 +#define BFQ_WEIGHT_CONVERSION_COEFF 10 + +#define BFQ_DEFAULT_QUEUE_IOPRIO 4 + +#define BFQ_WEIGHT_LEGACY_DFL 100 +#define BFQ_DEFAULT_GRP_IOPRIO 0 +#define BFQ_DEFAULT_GRP_CLASS IOPRIO_CLASS_BE + +/* + * Soft real-time applications are extremely more latency sensitive + * than interactive ones. Over-raise the weight of the former to + * privilege them against the latter. + */ +#define BFQ_SOFTRT_WEIGHT_FACTOR 100 + +struct bfq_entity; + +/** + * struct bfq_service_tree - per ioprio_class service tree. + * + * Each service tree represents a B-WF2Q+ scheduler on its own. Each + * ioprio_class has its own independent scheduler, and so its own + * bfq_service_tree. All the fields are protected by the queue lock + * of the containing bfqd. + */ +struct bfq_service_tree { + /* tree for active entities (i.e., those backlogged) */ + struct rb_root active; + /* tree for idle entities (i.e., not backlogged, with V <= F_i)*/ + struct rb_root idle; + + struct bfq_entity *first_idle; /* idle entity with minimum F_i */ + struct bfq_entity *last_idle; /* idle entity with maximum F_i */ + + u64 vtime; /* scheduler virtual time */ + /* scheduler weight sum; active and idle entities contribute to it */ + unsigned long wsum; +}; + +/** + * struct bfq_sched_data - multi-class scheduler. + * + * bfq_sched_data is the basic scheduler queue. It supports three + * ioprio_classes, and can be used either as a toplevel queue or as an + * intermediate queue in a hierarchical setup. + * + * The supported ioprio_classes are the same as in CFQ, in descending + * priority order, IOPRIO_CLASS_RT, IOPRIO_CLASS_BE, IOPRIO_CLASS_IDLE. + * Requests from higher priority queues are served before all the + * requests from lower priority queues; among requests of the same + * queue requests are served according to B-WF2Q+. + * + * The schedule is implemented by the service trees, plus the field + * @next_in_service, which points to the entity on the active trees + * that will be served next, if 1) no changes in the schedule occurs + * before the current in-service entity is expired, 2) the in-service + * queue becomes idle when it expires, and 3) if the entity pointed by + * in_service_entity is not a queue, then the in-service child entity + * of the entity pointed by in_service_entity becomes idle on + * expiration. This peculiar definition allows for the following + * optimization, not yet exploited: while a given entity is still in + * service, we already know which is the best candidate for next + * service among the other active entitities in the same parent + * entity. We can then quickly compare the timestamps of the + * in-service entity with those of such best candidate. + * + * All the fields are protected by the queue lock of the containing + * bfqd. + */ +struct bfq_sched_data { + struct bfq_entity *in_service_entity; /* entity in service */ + /* head-of-the-line entity in the scheduler (see comments above) */ + struct bfq_entity *next_in_service; + /* array of service trees, one per ioprio_class */ + struct bfq_service_tree service_tree[BFQ_IOPRIO_CLASSES]; + /* last time CLASS_IDLE was served */ + unsigned long bfq_class_idle_last_service; + +}; + +/** + * struct bfq_weight_counter - counter of the number of all active entities + * with a given weight. + */ +struct bfq_weight_counter { + unsigned int weight; /* weight of the entities this counter refers to */ + unsigned int num_active; /* nr of active entities with this weight */ + /* + * Weights tree member (see bfq_data's @queue_weights_tree and + * @group_weights_tree) + */ + struct rb_node weights_node; +}; + +/** + * struct bfq_entity - schedulable entity. + * + * A bfq_entity is used to represent either a bfq_queue (leaf node in the + * cgroup hierarchy) or a bfq_group into the upper level scheduler. Each + * entity belongs to the sched_data of the parent group in the cgroup + * hierarchy. Non-leaf entities have also their own sched_data, stored + * in @my_sched_data. + * + * Each entity stores independently its priority values; this would + * allow different weights on different devices, but this + * functionality is not exported to userspace by now. Priorities and + * weights are updated lazily, first storing the new values into the + * new_* fields, then setting the @prio_changed flag. As soon as + * there is a transition in the entity state that allows the priority + * update to take place the effective and the requested priority + * values are synchronized. + * + * Unless cgroups are used, the weight value is calculated from the + * ioprio to export the same interface as CFQ. When dealing with + * ``well-behaved'' queues (i.e., queues that do not spend too much + * time to consume their budget and have true sequential behavior, and + * when there are no external factors breaking anticipation) the + * relative weights at each level of the cgroups hierarchy should be + * guaranteed. All the fields are protected by the queue lock of the + * containing bfqd. + */ +struct bfq_entity { + struct rb_node rb_node; /* service_tree member */ + /* pointer to the weight counter associated with this entity */ + struct bfq_weight_counter *weight_counter; + + /* + * Flag, true if the entity is on a tree (either the active or + * the idle one of its service_tree) or is in service. + */ + bool on_st; + + u64 finish; /* B-WF2Q+ finish timestamp (aka F_i) */ + u64 start; /* B-WF2Q+ start timestamp (aka S_i) */ + + /* tree the entity is enqueued into; %NULL if not on a tree */ + struct rb_root *tree; + + /* + * minimum start time of the (active) subtree rooted at this + * entity; used for O(log N) lookups into active trees + */ + u64 min_start; + + /* amount of service received during the last service slot */ + int service; + + /* budget, used also to calculate F_i: F_i = S_i + @budget / @weight */ + int budget; + + unsigned int weight; /* weight of the queue */ + unsigned int new_weight; /* next weight if a change is in progress */ + + /* original weight, used to implement weight boosting */ + unsigned int orig_weight; + + /* parent entity, for hierarchical scheduling */ + struct bfq_entity *parent; + + /* + * For non-leaf nodes in the hierarchy, the associated + * scheduler queue, %NULL on leaf nodes. + */ + struct bfq_sched_data *my_sched_data; + /* the scheduler queue this entity belongs to */ + struct bfq_sched_data *sched_data; + + /* flag, set to request a weight, ioprio or ioprio_class change */ + int prio_changed; +}; + +struct bfq_group; + +/** + * struct bfq_queue - leaf schedulable entity. + * + * A bfq_queue is a leaf request queue; it can be associated with an + * io_context or more, if it is async or shared between cooperating + * processes. @cgroup holds a reference to the cgroup, to be sure that it + * does not disappear while a bfqq still references it (mostly to avoid + * races between request issuing and task migration followed by cgroup + * destruction). + * All the fields are protected by the queue lock of the containing bfqd. + */ +struct bfq_queue { + /* reference counter */ + int ref; + /* parent bfq_data */ + struct bfq_data *bfqd; + + /* current ioprio and ioprio class */ + unsigned short ioprio, ioprio_class; + /* next ioprio and ioprio class if a change is in progress */ + unsigned short new_ioprio, new_ioprio_class; + + /* + * Shared bfq_queue if queue is cooperating with one or more + * other queues. + */ + struct bfq_queue *new_bfqq; + /* request-position tree member (see bfq_group's @rq_pos_tree) */ + struct rb_node pos_node; + /* request-position tree root (see bfq_group's @rq_pos_tree) */ + struct rb_root *pos_root; + + /* sorted list of pending requests */ + struct rb_root sort_list; + /* if fifo isn't expired, next request to serve */ + struct request *next_rq; + /* number of sync and async requests queued */ + int queued[2]; + /* number of sync and async requests currently allocated */ + int allocated[2]; + /* number of pending metadata requests */ + int meta_pending; + /* fifo list of requests in sort_list */ + struct list_head fifo; + + /* entity representing this queue in the scheduler */ + struct bfq_entity entity; + + /* maximum budget allowed from the feedback mechanism */ + int max_budget; + /* budget expiration (in jiffies) */ + unsigned long budget_timeout; + + /* number of requests on the dispatch list or inside driver */ + int dispatched; + + unsigned int flags; /* status flags.*/ + + /* node for active/idle bfqq list inside parent bfqd */ + struct list_head bfqq_list; + + /* bit vector: a 1 for each seeky requests in history */ + u32 seek_history; + + /* node for the device's burst list */ + struct hlist_node burst_list_node; + + /* position of the last request enqueued */ + sector_t last_request_pos; + + /* Number of consecutive pairs of request completion and + * arrival, such that the queue becomes idle after the + * completion, but the next request arrives within an idle + * time slice; used only if the queue's IO_bound flag has been + * cleared. + */ + unsigned int requests_within_timer; + + /* pid of the process owning the queue, used for logging purposes */ + pid_t pid; + + /* + * Pointer to the bfq_io_cq owning the bfq_queue, set to %NULL + * if the queue is shared. + */ + struct bfq_io_cq *bic; + + /* current maximum weight-raising time for this queue */ + unsigned long wr_cur_max_time; + /* + * Minimum time instant such that, only if a new request is + * enqueued after this time instant in an idle @bfq_queue with + * no outstanding requests, then the task associated with the + * queue it is deemed as soft real-time (see the comments on + * the function bfq_bfqq_softrt_next_start()) + */ + unsigned long soft_rt_next_start; + /* + * Start time of the current weight-raising period if + * the @bfq-queue is being weight-raised, otherwise + * finish time of the last weight-raising period. + */ + unsigned long last_wr_start_finish; + /* factor by which the weight of this queue is multiplied */ + unsigned int wr_coeff; + /* + * Time of the last transition of the @bfq_queue from idle to + * backlogged. + */ + unsigned long last_idle_bklogged; + /* + * Cumulative service received from the @bfq_queue since the + * last transition from idle to backlogged. + */ + unsigned long service_from_backlogged; + /* + * Value of wr start time when switching to soft rt + */ + unsigned long wr_start_at_switch_to_srt; + + unsigned long split_time; /* time of last split */ +}; + +/** + * struct bfq_ttime - per process thinktime stats. + */ +struct bfq_ttime { + u64 last_end_request; /* completion time of last request */ + + u64 ttime_total; /* total process thinktime */ + unsigned long ttime_samples; /* number of thinktime samples */ + u64 ttime_mean; /* average process thinktime */ + +}; + +/** + * struct bfq_io_cq - per (request_queue, io_context) structure. + */ +struct bfq_io_cq { + /* associated io_cq structure */ + struct io_cq icq; /* must be the first member */ + /* array of two process queues, the sync and the async */ + struct bfq_queue *bfqq[2]; + /* associated @bfq_ttime struct */ + struct bfq_ttime ttime; + /* per (request_queue, blkcg) ioprio */ + int ioprio; +#ifdef BFQ_GROUP_IOSCHED_ENABLED + uint64_t blkcg_serial_nr; /* the current blkcg serial */ +#endif + + /* + * Snapshot of the has_short_time flag before merging; taken + * to remember its value while the queue is merged, so as to + * be able to restore it in case of split. + */ + bool saved_has_short_ttime; + /* + * Same purpose as the previous two fields for the I/O bound + * classification of a queue. + */ + bool saved_IO_bound; + + /* + * Same purpose as the previous fields for the value of the + * field keeping the queue's belonging to a large burst + */ + bool saved_in_large_burst; + /* + * True if the queue belonged to a burst list before its merge + * with another cooperating queue. + */ + bool was_in_burst_list; + + /* + * Similar to previous fields: save wr information. + */ + unsigned long saved_wr_coeff; + unsigned long saved_last_wr_start_finish; + unsigned long saved_wr_start_at_switch_to_srt; + unsigned int saved_wr_cur_max_time; +}; + +enum bfq_device_speed { + BFQ_BFQD_FAST, + BFQ_BFQD_SLOW, +}; + +/** + * struct bfq_data - per-device data structure. + * + * All the fields are protected by the @queue lock. + */ +struct bfq_data { + /* request queue for the device */ + struct request_queue *queue; + + /* root bfq_group for the device */ + struct bfq_group *root_group; + + /* + * rbtree of weight counters of @bfq_queues, sorted by + * weight. Used to keep track of whether all @bfq_queues have + * the same weight. The tree contains one counter for each + * distinct weight associated to some active and not + * weight-raised @bfq_queue (see the comments to the functions + * bfq_weights_tree_[add|remove] for further details). + */ + struct rb_root queue_weights_tree; + /* + * rbtree of non-queue @bfq_entity weight counters, sorted by + * weight. Used to keep track of whether all @bfq_groups have + * the same weight. The tree contains one counter for each + * distinct weight associated to some active @bfq_group (see + * the comments to the functions bfq_weights_tree_[add|remove] + * for further details). + */ + struct rb_root group_weights_tree; + + /* + * Number of bfq_queues containing requests (including the + * queue in service, even if it is idling). + */ + int busy_queues; + /* number of weight-raised busy @bfq_queues */ + int wr_busy_queues; + /* number of queued requests */ + int queued; + /* number of requests dispatched and waiting for completion */ + int rq_in_driver; + + /* + * Maximum number of requests in driver in the last + * @hw_tag_samples completed requests. + */ + int max_rq_in_driver; + /* number of samples used to calculate hw_tag */ + int hw_tag_samples; + /* flag set to one if the driver is showing a queueing behavior */ + int hw_tag; + + /* number of budgets assigned */ + int budgets_assigned; + + /* + * Timer set when idling (waiting) for the next request from + * the queue in service. + */ + struct hrtimer idle_slice_timer; + /* delayed work to restart dispatching on the request queue */ + struct work_struct unplug_work; + + /* bfq_queue in service */ + struct bfq_queue *in_service_queue; + /* bfq_io_cq (bic) associated with the @in_service_queue */ + struct bfq_io_cq *in_service_bic; + + /* on-disk position of the last served request */ + sector_t last_position; + + /* time of last request completion (ns) */ + u64 last_completion; + + /* time of first rq dispatch in current observation interval (ns) */ + u64 first_dispatch; + /* time of last rq dispatch in current observation interval (ns) */ + u64 last_dispatch; + + /* beginning of the last budget */ + ktime_t last_budget_start; + /* beginning of the last idle slice */ + ktime_t last_idling_start; + + /* number of samples in current observation interval */ + int peak_rate_samples; + /* num of samples of seq dispatches in current observation interval */ + u32 sequential_samples; + /* total num of sectors transferred in current observation interval */ + u64 tot_sectors_dispatched; + /* max rq size seen during current observation interval (sectors) */ + u32 last_rq_max_size; + /* time elapsed from first dispatch in current observ. interval (us) */ + u64 delta_from_first; + /* current estimate of device peak rate */ + u32 peak_rate; + + /* maximum budget allotted to a bfq_queue before rescheduling */ + int bfq_max_budget; + + /* list of all the bfq_queues active on the device */ + struct list_head active_list; + /* list of all the bfq_queues idle on the device */ + struct list_head idle_list; + + /* + * Timeout for async/sync requests; when it fires, requests + * are served in fifo order. + */ + u64 bfq_fifo_expire[2]; + /* weight of backward seeks wrt forward ones */ + unsigned int bfq_back_penalty; + /* maximum allowed backward seek */ + unsigned int bfq_back_max; + /* maximum idling time */ + u32 bfq_slice_idle; + + /* user-configured max budget value (0 for auto-tuning) */ + int bfq_user_max_budget; + /* + * Timeout for bfq_queues to consume their budget; used to + * prevent seeky queues from imposing long latencies to + * sequential or quasi-sequential ones (this also implies that + * seeky queues cannot receive guarantees in the service + * domain; after a timeout they are charged for the time they + * have been in service, to preserve fairness among them, but + * without service-domain guarantees). + */ + unsigned int bfq_timeout; + + /* + * Number of consecutive requests that must be issued within + * the idle time slice to set again idling to a queue which + * was marked as non-I/O-bound (see the definition of the + * IO_bound flag for further details). + */ + unsigned int bfq_requests_within_timer; + + /* + * Force device idling whenever needed to provide accurate + * service guarantees, without caring about throughput + * issues. CAVEAT: this may even increase latencies, in case + * of useless idling for processes that did stop doing I/O. + */ + bool strict_guarantees; + + /* + * Last time at which a queue entered the current burst of + * queues being activated shortly after each other; for more + * details about this and the following parameters related to + * a burst of activations, see the comments on the function + * bfq_handle_burst. + */ + unsigned long last_ins_in_burst; + /* + * Reference time interval used to decide whether a queue has + * been activated shortly after @last_ins_in_burst. + */ + unsigned long bfq_burst_interval; + /* number of queues in the current burst of queue activations */ + int burst_size; + + /* common parent entity for the queues in the burst */ + struct bfq_entity *burst_parent_entity; + /* Maximum burst size above which the current queue-activation + * burst is deemed as 'large'. + */ + unsigned long bfq_large_burst_thresh; + /* true if a large queue-activation burst is in progress */ + bool large_burst; + /* + * Head of the burst list (as for the above fields, more + * details in the comments on the function bfq_handle_burst). + */ + struct hlist_head burst_list; + + /* if set to true, low-latency heuristics are enabled */ + bool low_latency; + /* + * Maximum factor by which the weight of a weight-raised queue + * is multiplied. + */ + unsigned int bfq_wr_coeff; + /* maximum duration of a weight-raising period (jiffies) */ + unsigned int bfq_wr_max_time; + + /* Maximum weight-raising duration for soft real-time processes */ + unsigned int bfq_wr_rt_max_time; + /* + * Minimum idle period after which weight-raising may be + * reactivated for a queue (in jiffies). + */ + unsigned int bfq_wr_min_idle_time; + /* + * Minimum period between request arrivals after which + * weight-raising may be reactivated for an already busy async + * queue (in jiffies). + */ + unsigned long bfq_wr_min_inter_arr_async; + + /* Max service-rate for a soft real-time queue, in sectors/sec */ + unsigned int bfq_wr_max_softrt_rate; + /* + * Cached value of the product R*T, used for computing the + * maximum duration of weight raising automatically. + */ + u64 RT_prod; + /* device-speed class for the low-latency heuristic */ + enum bfq_device_speed device_speed; + + /* fallback dummy bfqq for extreme OOM conditions */ + struct bfq_queue oom_bfqq; +}; + +enum bfqq_state_flags { + BFQ_BFQQ_FLAG_just_created = 0, /* queue just allocated */ + BFQ_BFQQ_FLAG_busy, /* has requests or is in service */ + BFQ_BFQQ_FLAG_wait_request, /* waiting for a request */ + BFQ_BFQQ_FLAG_non_blocking_wait_rq, /* + * waiting for a request + * without idling the device + */ + BFQ_BFQQ_FLAG_must_alloc, /* must be allowed rq alloc */ + BFQ_BFQQ_FLAG_fifo_expire, /* FIFO checked in this slice */ + BFQ_BFQQ_FLAG_has_short_ttime, /* queue has a short think time */ + BFQ_BFQQ_FLAG_sync, /* synchronous queue */ + BFQ_BFQQ_FLAG_IO_bound, /* + * bfqq has timed-out at least once + * having consumed at most 2/10 of + * its budget + */ + BFQ_BFQQ_FLAG_in_large_burst, /* + * bfqq activated in a large burst, + * see comments to bfq_handle_burst. + */ + BFQ_BFQQ_FLAG_softrt_update, /* + * may need softrt-next-start + * update + */ + BFQ_BFQQ_FLAG_coop, /* bfqq is shared */ + BFQ_BFQQ_FLAG_split_coop /* shared bfqq will be split */ +}; + +#define BFQ_BFQQ_FNS(name) \ +static void bfq_mark_bfqq_##name(struct bfq_queue *bfqq) \ +{ \ + (bfqq)->flags |= (1 << BFQ_BFQQ_FLAG_##name); \ +} \ +static void bfq_clear_bfqq_##name(struct bfq_queue *bfqq) \ +{ \ + (bfqq)->flags &= ~(1 << BFQ_BFQQ_FLAG_##name); \ +} \ +static int bfq_bfqq_##name(const struct bfq_queue *bfqq) \ +{ \ + return ((bfqq)->flags & (1 << BFQ_BFQQ_FLAG_##name)) != 0; \ +} + +BFQ_BFQQ_FNS(just_created); +BFQ_BFQQ_FNS(busy); +BFQ_BFQQ_FNS(wait_request); +BFQ_BFQQ_FNS(non_blocking_wait_rq); +BFQ_BFQQ_FNS(must_alloc); +BFQ_BFQQ_FNS(fifo_expire); +BFQ_BFQQ_FNS(has_short_ttime); +BFQ_BFQQ_FNS(sync); +BFQ_BFQQ_FNS(IO_bound); +BFQ_BFQQ_FNS(in_large_burst); +BFQ_BFQQ_FNS(coop); +BFQ_BFQQ_FNS(split_coop); +BFQ_BFQQ_FNS(softrt_update); +#undef BFQ_BFQQ_FNS + +/* Logging facilities. */ +#ifdef CONFIG_BFQ_REDIRECT_TO_CONSOLE + +static const char *checked_dev_name(const struct device *dev) +{ + static const char nodev[] = "nodev"; + + if (dev) + return dev_name(dev); + + return nodev; +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq); +static struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) do { \ + char __pbuf[128]; \ + \ + assert_spin_locked((bfqd)->queue->queue_lock); \ + blkg_path(bfqg_to_blkg(bfqq_group(bfqq)), __pbuf, sizeof(__pbuf)); \ + pr_crit("%s bfq%d%c %s " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + (bfqq)->pid, \ + bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + __pbuf, ##args); \ +} while (0) + +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do { \ + char __pbuf[128]; \ + \ + blkg_path(bfqg_to_blkg(bfqg), __pbuf, sizeof(__pbuf)); \ + pr_crit("%s %s " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + __pbuf, ##args); \ +} while (0) + +#else /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) \ + pr_crit("%s bfq%d%c " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + (bfqq)->pid, bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + ##args) +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do {} while (0) + +#endif /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log(bfqd, fmt, args...) \ + pr_crit("%s bfq " fmt "\n", \ + checked_dev_name((bfqd)->queue->backing_dev_info->dev), \ + ##args) + +#else /* CONFIG_BFQ_REDIRECT_TO_CONSOLE */ + +#if !defined(CONFIG_BLK_DEV_IO_TRACE) + +/* Avoid possible "unused-variable" warning. See commit message. */ + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) ((void) (bfqq)) + +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) ((void) (bfqg)) + +#define bfq_log(bfqd, fmt, args...) do {} while (0) + +#else /* CONFIG_BLK_DEV_IO_TRACE */ + +#include + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static struct bfq_group *bfqq_group(struct bfq_queue *bfqq); +static struct blkcg_gq *bfqg_to_blkg(struct bfq_group *bfqg); + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) do { \ + char __pbuf[128]; \ + \ + assert_spin_locked((bfqd)->queue->queue_lock); \ + blkg_path(bfqg_to_blkg(bfqq_group(bfqq)), __pbuf, sizeof(__pbuf)); \ + blk_add_trace_msg((bfqd)->queue, "bfq%d%c %s " fmt, \ + (bfqq)->pid, \ + bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + __pbuf, ##args); \ +} while (0) + +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do { \ + char __pbuf[128]; \ + \ + blkg_path(bfqg_to_blkg(bfqg), __pbuf, sizeof(__pbuf)); \ + blk_add_trace_msg((bfqd)->queue, "%s " fmt, __pbuf, ##args); \ +} while (0) + +#else /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log_bfqq(bfqd, bfqq, fmt, args...) \ + blk_add_trace_msg((bfqd)->queue, "bfq%d%c " fmt, (bfqq)->pid, \ + bfq_bfqq_sync((bfqq)) ? 'S' : 'A', \ + ##args) +#define bfq_log_bfqg(bfqd, bfqg, fmt, args...) do {} while (0) + +#endif /* BFQ_GROUP_IOSCHED_ENABLED */ + +#define bfq_log(bfqd, fmt, args...) \ + blk_add_trace_msg((bfqd)->queue, "bfq " fmt, ##args) + +#endif /* CONFIG_BLK_DEV_IO_TRACE */ +#endif /* CONFIG_BFQ_REDIRECT_TO_CONSOLE */ + +/* Expiration reasons. */ +enum bfqq_expiration { + BFQ_BFQQ_TOO_IDLE = 0, /* + * queue has been idling for + * too long + */ + BFQ_BFQQ_BUDGET_TIMEOUT, /* budget took too long to be used */ + BFQ_BFQQ_BUDGET_EXHAUSTED, /* budget consumed */ + BFQ_BFQQ_NO_MORE_REQUESTS, /* the queue has no more requests */ + BFQ_BFQQ_PREEMPTED /* preemption in progress */ +}; + + +struct bfqg_stats { +#ifdef BFQ_GROUP_IOSCHED_ENABLED + /* number of ios merged */ + struct blkg_rwstat merged; + /* total time spent on device in ns, may not be accurate w/ queueing */ + struct blkg_rwstat service_time; + /* total time spent waiting in scheduler queue in ns */ + struct blkg_rwstat wait_time; + /* number of IOs queued up */ + struct blkg_rwstat queued; + /* total disk time and nr sectors dispatched by this group */ + struct blkg_stat time; + /* sum of number of ios queued across all samples */ + struct blkg_stat avg_queue_size_sum; + /* count of samples taken for average */ + struct blkg_stat avg_queue_size_samples; + /* how many times this group has been removed from service tree */ + struct blkg_stat dequeue; + /* total time spent waiting for it to be assigned a timeslice. */ + struct blkg_stat group_wait_time; + /* time spent idling for this blkcg_gq */ + struct blkg_stat idle_time; + /* total time with empty current active q with other requests queued */ + struct blkg_stat empty_time; + /* fields after this shouldn't be cleared on stat reset */ + uint64_t start_group_wait_time; + uint64_t start_idle_time; + uint64_t start_empty_time; + uint16_t flags; +#endif +}; + +#ifdef BFQ_GROUP_IOSCHED_ENABLED +/* + * struct bfq_group_data - per-blkcg storage for the blkio subsystem. + * + * @ps: @blkcg_policy_storage that this structure inherits + * @weight: weight of the bfq_group + */ +struct bfq_group_data { + /* must be the first member */ + struct blkcg_policy_data pd; + + unsigned int weight; +}; + +/** + * struct bfq_group - per (device, cgroup) data structure. + * @entity: schedulable entity to insert into the parent group sched_data. + * @sched_data: own sched_data, to contain child entities (they may be + * both bfq_queues and bfq_groups). + * @bfqd: the bfq_data for the device this group acts upon. + * @async_bfqq: array of async queues for all the tasks belonging to + * the group, one queue per ioprio value per ioprio_class, + * except for the idle class that has only one queue. + * @async_idle_bfqq: async queue for the idle class (ioprio is ignored). + * @my_entity: pointer to @entity, %NULL for the toplevel group; used + * to avoid too many special cases during group creation/ + * migration. + * @active_entities: number of active entities belonging to the group; + * unused for the root group. Used to know whether there + * are groups with more than one active @bfq_entity + * (see the comments to the function + * bfq_bfqq_may_idle()). + * @rq_pos_tree: rbtree sorted by next_request position, used when + * determining if two or more queues have interleaving + * requests (see bfq_find_close_cooperator()). + * + * Each (device, cgroup) pair has its own bfq_group, i.e., for each cgroup + * there is a set of bfq_groups, each one collecting the lower-level + * entities belonging to the group that are acting on the same device. + * + * Locking works as follows: + * o @bfqd is protected by the queue lock, RCU is used to access it + * from the readers. + * o All the other fields are protected by the @bfqd queue lock. + */ +struct bfq_group { + /* must be the first member */ + struct blkg_policy_data pd; + + struct bfq_entity entity; + struct bfq_sched_data sched_data; + + void *bfqd; + + struct bfq_queue *async_bfqq[2][IOPRIO_BE_NR]; + struct bfq_queue *async_idle_bfqq; + + struct bfq_entity *my_entity; + + int active_entities; + + struct rb_root rq_pos_tree; + + struct bfqg_stats stats; +}; + +#else +struct bfq_group { + struct bfq_sched_data sched_data; + + struct bfq_queue *async_bfqq[2][IOPRIO_BE_NR]; + struct bfq_queue *async_idle_bfqq; + + struct rb_root rq_pos_tree; +}; +#endif + +static struct bfq_queue *bfq_entity_to_bfqq(struct bfq_entity *entity); + +static unsigned int bfq_class_idx(struct bfq_entity *entity) +{ + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + + return bfqq ? bfqq->ioprio_class - 1 : + BFQ_DEFAULT_GRP_CLASS - 1; +} + +static struct bfq_service_tree * +bfq_entity_service_tree(struct bfq_entity *entity) +{ + struct bfq_sched_data *sched_data = entity->sched_data; + struct bfq_queue *bfqq = bfq_entity_to_bfqq(entity); + unsigned int idx = bfq_class_idx(entity); + + BUG_ON(idx >= BFQ_IOPRIO_CLASSES); + BUG_ON(sched_data == NULL); + + if (bfqq) + bfq_log_bfqq(bfqq->bfqd, bfqq, + "entity_service_tree %p %d", + sched_data->service_tree + idx, idx); +#ifdef BFQ_GROUP_IOSCHED_ENABLED + else { + struct bfq_group *bfqg = + container_of(entity, struct bfq_group, entity); + + bfq_log_bfqg((struct bfq_data *)bfqg->bfqd, bfqg, + "entity_service_tree %p %d", + sched_data->service_tree + idx, idx); + } +#endif + return sched_data->service_tree + idx; +} + +static struct bfq_queue *bic_to_bfqq(struct bfq_io_cq *bic, bool is_sync) +{ + return bic->bfqq[is_sync]; +} + +static void bic_set_bfqq(struct bfq_io_cq *bic, struct bfq_queue *bfqq, + bool is_sync) +{ + bic->bfqq[is_sync] = bfqq; +} + +static struct bfq_data *bic_to_bfqd(struct bfq_io_cq *bic) +{ + return bic->icq.q->elevator->elevator_data; +} + +#ifdef BFQ_GROUP_IOSCHED_ENABLED + +static struct bfq_group *bfq_bfqq_to_bfqg(struct bfq_queue *bfqq) +{ + struct bfq_entity *group_entity = bfqq->entity.parent; + + if (!group_entity) + group_entity = &bfqq->bfqd->root_group->entity; + + return container_of(group_entity, struct bfq_group, entity); +} + +#else + +static struct bfq_group *bfq_bfqq_to_bfqg(struct bfq_queue *bfqq) +{ + return bfqq->bfqd->root_group; +} + +#endif + +static void bfq_check_ioprio_change(struct bfq_io_cq *bic, struct bio *bio); +static void bfq_put_queue(struct bfq_queue *bfqq); +static void bfq_dispatch_insert(struct request_queue *q, struct request *rq); +static struct bfq_queue *bfq_get_queue(struct bfq_data *bfqd, + struct bio *bio, bool is_sync, + struct bfq_io_cq *bic); +static void bfq_end_wr_async_queues(struct bfq_data *bfqd, + struct bfq_group *bfqg); +#ifdef BFQ_GROUP_IOSCHED_ENABLED +static void bfq_put_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg); +#endif +static void bfq_exit_bfqq(struct bfq_data *bfqd, struct bfq_queue *bfqq); + +#endif /* _BFQ_H */ diff --git a/block/elevator.c b/block/elevator.c index 4bb2f0c..3e9e54d 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -229,7 +229,11 @@ int elevator_init(struct request_queue *q, char *name) */ if (q->mq_ops) { if (q->nr_hw_queues == 1) + #if defined(CONFIG_PCK_INTERACTIVE) && defined(CONFIG_IOSCHED_BFQ) + e = elevator_get("bfq", false); + #else e = elevator_get("mq-deadline", false); + #endif if (!e) return 0; } else diff --git a/drivers/cpufreq/cpufreq_ondemand.c b/drivers/cpufreq/cpufreq_ondemand.c index 3937acf..2fba66f 100644 --- a/drivers/cpufreq/cpufreq_ondemand.c +++ b/drivers/cpufreq/cpufreq_ondemand.c @@ -21,10 +21,18 @@ #include "cpufreq_ondemand.h" /* On-demand governor macros */ -#define DEF_FREQUENCY_UP_THRESHOLD (80) +#if defined(CONFIG_PCK_INTERACTIVE) && defined(CONFIG_SCHED_MUQSS) + #define DEF_FREQUENCY_UP_THRESHOLD (45) + #define MICRO_FREQUENCY_UP_THRESHOLD (45) +#elif defined(CONFIG_PCK_INTERACTIVE) + #define DEF_FREQUENCY_UP_THRESHOLD (80) + #define MICRO_FREQUENCY_UP_THRESHOLD (85) +#else + #define DEF_FREQUENCY_UP_THRESHOLD (80) + #define MICRO_FREQUENCY_UP_THRESHOLD (95) +#endif #define DEF_SAMPLING_DOWN_FACTOR (1) #define MAX_SAMPLING_DOWN_FACTOR (100000) -#define MICRO_FREQUENCY_UP_THRESHOLD (95) #define MICRO_FREQUENCY_MIN_SAMPLE_RATE (10000) #define MIN_FREQUENCY_UP_THRESHOLD (1) #define MAX_FREQUENCY_UP_THRESHOLD (100) diff --git a/drivers/input/mouse/synaptics.c b/drivers/input/mouse/synaptics.c index 5af0b7d..f83df70 100644 --- a/drivers/input/mouse/synaptics.c +++ b/drivers/input/mouse/synaptics.c @@ -1315,7 +1315,9 @@ static void set_input_params(struct psmouse *psmouse, /* Clickpads report only left button */ __clear_bit(BTN_RIGHT, dev->keybit); __clear_bit(BTN_MIDDLE, dev->keybit); - } + } else if (SYN_CAP_CLICKPAD2BTN(info->ext_cap_0c) || + SYN_CAP_CLICKPAD2BTN2(info->ext_cap_0c)) + __set_bit(INPUT_PROP_BUTTONPAD, dev->propbit); } static ssize_t synaptics_show_disable_gesture(struct psmouse *psmouse, diff --git a/drivers/input/mouse/synaptics.h b/drivers/input/mouse/synaptics.h index fc00e00..4cfbeec 100644 --- a/drivers/input/mouse/synaptics.h +++ b/drivers/input/mouse/synaptics.h @@ -86,6 +86,7 @@ */ #define SYN_CAP_CLICKPAD(ex0c) ((ex0c) & BIT(20)) /* 1-button ClickPad */ #define SYN_CAP_CLICKPAD2BTN(ex0c) ((ex0c) & BIT(8)) /* 2-button ClickPad */ +#define SYN_CAP_CLICKPAD2BTN2(ex0c) ((ex0c) & BIT(21)) /* 2-button ClickPad */ #define SYN_CAP_MAX_DIMENSIONS(ex0c) ((ex0c) & BIT(17)) #define SYN_CAP_MIN_DIMENSIONS(ex0c) ((ex0c) & BIT(13)) #define SYN_CAP_ADV_GESTURE(ex0c) ((ex0c) & BIT(19)) diff --git a/drivers/macintosh/Kconfig b/drivers/macintosh/Kconfig index 97a420c..c8621e9 100644 --- a/drivers/macintosh/Kconfig +++ b/drivers/macintosh/Kconfig @@ -159,6 +159,13 @@ config INPUT_ADBHID If unsure, say Y. +config ADB_TRACKPAD_ABSOLUTE + bool "Enable absolute mode for adb trackpads" + depends on INPUT_ADBHID + help + Enable absolute mode in adb-base trackpads. This feature adds + compatibility with synaptics Xorg / Xfree drivers. + config MAC_EMUMOUSEBTN tristate "Support for mouse button 2+3 emulation" depends on SYSCTL && INPUT diff --git a/drivers/macintosh/adbhid.c b/drivers/macintosh/adbhid.c index 09d72bb..8d23b27 100644 --- a/drivers/macintosh/adbhid.c +++ b/drivers/macintosh/adbhid.c @@ -261,6 +261,15 @@ static struct adb_ids buttons_ids; #define ADBMOUSE_MS_A3 8 /* Mouse systems A3 trackball (handler 3) */ #define ADBMOUSE_MACALLY2 9 /* MacAlly 2-button mouse */ +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE +#define ABS_XMIN 310 +#define ABS_XMAX 1700 +#define ABS_YMIN 200 +#define ABS_YMAX 1000 +#define ABS_ZMIN 0 +#define ABS_ZMAX 55 +#endif + static void adbhid_keyboard_input(unsigned char *data, int nb, int apoll) { @@ -405,6 +414,9 @@ static void adbhid_mouse_input(unsigned char *data, int nb, int autopoll) { int id = (data[0] >> 4) & 0x0f; +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE + int btn = 0; int x_axis = 0; int y_axis = 0; int z_axis = 0; +#endif if (!adbhid[id]) { printk(KERN_ERR "ADB HID on ID %d not yet registered\n", id); @@ -436,6 +448,17 @@ adbhid_mouse_input(unsigned char *data, int nb, int autopoll) high bits of y-axis motion. XY is additional high bits of x-axis motion. + For ADB Absolute motion protocol the data array will contain the + following values: + + BITS COMMENTS + data[0] = dddd 1100 ADB command: Talk, register 0, for device dddd. + data[1] = byyy yyyy Left button and y-axis motion. + data[2] = bxxx xxxx Second button and x-axis motion. + data[3] = 1yyy 1xxx Half bits of y-axis and x-axis motion. + data[4] = 1yyy 1xxx Higher bits of y-axis and x-axis motion. + data[5] = 1zzz 1zzz Higher and lower bits of z-pressure. + MacAlly 2-button mouse protocol. For MacAlly 2-button mouse protocol the data array will contain the @@ -458,8 +481,17 @@ adbhid_mouse_input(unsigned char *data, int nb, int autopoll) switch (adbhid[id]->mouse_kind) { case ADBMOUSE_TRACKPAD: +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE + x_axis = (data[2] & 0x7f) | ((data[3] & 0x07) << 7) | + ((data[4] & 0x07) << 10); + y_axis = (data[1] & 0x7f) | ((data[3] & 0x70) << 3) | + ((data[4] & 0x70) << 6); + z_axis = (data[5] & 0x07) | ((data[5] & 0x70) >> 1); + btn = (!(data[1] >> 7)) & 1; +#else data[1] = (data[1] & 0x7f) | ((data[1] & data[2]) & 0x80); data[2] = data[2] | 0x80; +#endif break; case ADBMOUSE_MICROSPEED: data[1] = (data[1] & 0x7f) | ((data[3] & 0x01) << 7); @@ -485,17 +517,39 @@ adbhid_mouse_input(unsigned char *data, int nb, int autopoll) break; } - input_report_key(adbhid[id]->input, BTN_LEFT, !((data[1] >> 7) & 1)); - input_report_key(adbhid[id]->input, BTN_MIDDLE, !((data[2] >> 7) & 1)); +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE + if ( adbhid[id]->mouse_kind == ADBMOUSE_TRACKPAD ) { - if (nb >= 4 && adbhid[id]->mouse_kind != ADBMOUSE_TRACKPAD) - input_report_key(adbhid[id]->input, BTN_RIGHT, !((data[3] >> 7) & 1)); + if(z_axis > 30) input_report_key(adbhid[id]->input, BTN_TOUCH, 1); + if(z_axis < 25) input_report_key(adbhid[id]->input, BTN_TOUCH, 0); - input_report_rel(adbhid[id]->input, REL_X, - ((data[2]&0x7f) < 64 ? (data[2]&0x7f) : (data[2]&0x7f)-128 )); - input_report_rel(adbhid[id]->input, REL_Y, - ((data[1]&0x7f) < 64 ? (data[1]&0x7f) : (data[1]&0x7f)-128 )); + if(z_axis > 0){ + input_report_abs(adbhid[id]->input, ABS_X, x_axis); + input_report_abs(adbhid[id]->input, ABS_Y, y_axis); + input_report_key(adbhid[id]->input, BTN_TOOL_FINGER, 1); + input_report_key(adbhid[id]->input, ABS_TOOL_WIDTH, 5); + } else { + input_report_key(adbhid[id]->input, BTN_TOOL_FINGER, 0); + input_report_key(adbhid[id]->input, ABS_TOOL_WIDTH, 0); + } + + input_report_abs(adbhid[id]->input, ABS_PRESSURE, z_axis); + input_report_key(adbhid[id]->input, BTN_LEFT, btn); + } else { +#endif + input_report_key(adbhid[id]->input, BTN_LEFT, !((data[1] >> 7) & 1)); + input_report_key(adbhid[id]->input, BTN_MIDDLE, !((data[2] >> 7) & 1)); + + if (nb >= 4 && adbhid[id]->mouse_kind != ADBMOUSE_TRACKPAD) + input_report_key(adbhid[id]->input, BTN_RIGHT, !((data[3] >> 7) & 1)); + input_report_rel(adbhid[id]->input, REL_X, + ((data[2]&0x7f) < 64 ? (data[2]&0x7f) : (data[2]&0x7f)-128 )); + input_report_rel(adbhid[id]->input, REL_Y, + ((data[1]&0x7f) < 64 ? (data[1]&0x7f) : (data[1]&0x7f)-128 )); +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE + } +#endif input_sync(adbhid[id]->input); } @@ -849,6 +903,15 @@ adbhid_input_register(int id, int default_id, int original_handler_id, input_dev->keybit[BIT_WORD(BTN_MOUSE)] = BIT_MASK(BTN_LEFT) | BIT_MASK(BTN_MIDDLE) | BIT_MASK(BTN_RIGHT); input_dev->relbit[0] = BIT_MASK(REL_X) | BIT_MASK(REL_Y); +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE + set_bit(EV_ABS, input_dev->evbit); + input_set_abs_params(input_dev, ABS_X, ABS_XMIN, ABS_XMAX, 0, 0); + input_set_abs_params(input_dev, ABS_Y, ABS_YMIN, ABS_YMAX, 0, 0); + input_set_abs_params(input_dev, ABS_PRESSURE, ABS_ZMIN, ABS_ZMAX, 0, 0); + set_bit(BTN_TOUCH, input_dev->keybit); + set_bit(BTN_TOOL_FINGER, input_dev->keybit); + set_bit(ABS_TOOL_WIDTH, input_dev->absbit); +#endif break; case ADB_MISC: @@ -1132,7 +1195,11 @@ init_trackpad(int id) r1_buffer[3], r1_buffer[4], r1_buffer[5], +#ifdef CONFIG_ADB_TRACKPAD_ABSOLUTE + 0x00, /* Enable absolute mode */ +#else 0x03, /*r1_buffer[6],*/ +#endif r1_buffer[7]); /* Without this flush, the trackpad may be locked up */ diff --git a/drivers/platform/x86/Kconfig b/drivers/platform/x86/Kconfig index 80b8795..6685d38 100644 --- a/drivers/platform/x86/Kconfig +++ b/drivers/platform/x86/Kconfig @@ -511,9 +511,28 @@ config THINKPAD_ACPI_HOTKEY_POLL If you are not sure, say Y here. The driver enables polling only if it is strictly necessary to do so. +config THINKPAD_EC + tristate + ---help--- + This is a low-level driver for accessing the ThinkPad H8S embedded + controller over the LPC bus (not to be confused with the ACPI Embedded + Controller interface). + +config TP_SMAPI + tristate "ThinkPad SMAPI Support" + select THINKPAD_EC + default n + help + This adds SMAPI support on Lenovo/IBM ThinkPads, for features such + as battery charging control. For more information about this driver + see . + + If you have a Lenovo/IBM ThinkPad laptop, say Y or M here. + config SENSORS_HDAPS tristate "Thinkpad Hard Drive Active Protection System (hdaps)" depends on INPUT + select THINKPAD_EC select INPUT_POLLDEV default n help diff --git a/drivers/platform/x86/Makefile b/drivers/platform/x86/Makefile index 91cec17..d0f6e81 100644 --- a/drivers/platform/x86/Makefile +++ b/drivers/platform/x86/Makefile @@ -28,6 +28,8 @@ obj-$(CONFIG_TC1100_WMI) += tc1100-wmi.o obj-$(CONFIG_SONY_LAPTOP) += sony-laptop.o obj-$(CONFIG_IDEAPAD_LAPTOP) += ideapad-laptop.o obj-$(CONFIG_THINKPAD_ACPI) += thinkpad_acpi.o +obj-$(CONFIG_THINKPAD_EC) += thinkpad_ec.o +obj-$(CONFIG_TP_SMAPI) += tp_smapi.o obj-$(CONFIG_SENSORS_HDAPS) += hdaps.o obj-$(CONFIG_FUJITSU_LAPTOP) += fujitsu-laptop.o obj-$(CONFIG_FUJITSU_TABLET) += fujitsu-tablet.o diff --git a/drivers/platform/x86/hdaps.c b/drivers/platform/x86/hdaps.c index 458e6c9..dadfb99 100644 --- a/drivers/platform/x86/hdaps.c +++ b/drivers/platform/x86/hdaps.c @@ -30,266 +30,384 @@ #include #include -#include +#include #include -#include #include #include #include #include -#include - -#define HDAPS_LOW_PORT 0x1600 /* first port used by hdaps */ -#define HDAPS_NR_PORTS 0x30 /* number of ports: 0x1600 - 0x162f */ - -#define HDAPS_PORT_STATE 0x1611 /* device state */ -#define HDAPS_PORT_YPOS 0x1612 /* y-axis position */ -#define HDAPS_PORT_XPOS 0x1614 /* x-axis position */ -#define HDAPS_PORT_TEMP1 0x1616 /* device temperature, in Celsius */ -#define HDAPS_PORT_YVAR 0x1617 /* y-axis variance (what is this?) */ -#define HDAPS_PORT_XVAR 0x1619 /* x-axis variance (what is this?) */ -#define HDAPS_PORT_TEMP2 0x161b /* device temperature (again?) */ -#define HDAPS_PORT_UNKNOWN 0x161c /* what is this? */ -#define HDAPS_PORT_KMACT 0x161d /* keyboard or mouse activity */ - -#define STATE_FRESH 0x50 /* accelerometer data is fresh */ +#include +#include +#include + +/* Embedded controller accelerometer read command and its result: */ +static const struct thinkpad_ec_row ec_accel_args = + { .mask = 0x0001, .val = {0x11} }; +#define EC_ACCEL_IDX_READOUTS 0x1 /* readouts included in this read */ + /* First readout, if READOUTS>=1: */ +#define EC_ACCEL_IDX_YPOS1 0x2 /* y-axis position word */ +#define EC_ACCEL_IDX_XPOS1 0x4 /* x-axis position word */ +#define EC_ACCEL_IDX_TEMP1 0x6 /* device temperature in Celsius */ + /* Second readout, if READOUTS>=2: */ +#define EC_ACCEL_IDX_XPOS2 0x7 /* y-axis position word */ +#define EC_ACCEL_IDX_YPOS2 0x9 /* x-axis position word */ +#define EC_ACCEL_IDX_TEMP2 0xb /* device temperature in Celsius */ +#define EC_ACCEL_IDX_QUEUED 0xc /* Number of queued readouts left */ +#define EC_ACCEL_IDX_KMACT 0xd /* keyboard or mouse activity */ +#define EC_ACCEL_IDX_RETVAL 0xf /* command return value, good=0x00 */ #define KEYBD_MASK 0x20 /* set if keyboard activity */ #define MOUSE_MASK 0x40 /* set if mouse activity */ -#define KEYBD_ISSET(n) (!! (n & KEYBD_MASK)) /* keyboard used? */ -#define MOUSE_ISSET(n) (!! (n & MOUSE_MASK)) /* mouse used? */ -#define INIT_TIMEOUT_MSECS 4000 /* wait up to 4s for device init ... */ -#define INIT_WAIT_MSECS 200 /* ... in 200ms increments */ +#define READ_TIMEOUT_MSECS 100 /* wait this long for device read */ +#define RETRY_MSECS 3 /* retry delay */ -#define HDAPS_POLL_INTERVAL 50 /* poll for input every 1/20s (50 ms)*/ #define HDAPS_INPUT_FUZZ 4 /* input event threshold */ #define HDAPS_INPUT_FLAT 4 - -#define HDAPS_X_AXIS (1 << 0) -#define HDAPS_Y_AXIS (1 << 1) -#define HDAPS_BOTH_AXES (HDAPS_X_AXIS | HDAPS_Y_AXIS) - +#define KMACT_REMEMBER_PERIOD (HZ/10) /* keyboard/mouse persistance */ + +/* Input IDs */ +#define HDAPS_INPUT_VENDOR PCI_VENDOR_ID_IBM +#define HDAPS_INPUT_PRODUCT 0x5054 /* "TP", shared with thinkpad_acpi */ +#define HDAPS_INPUT_JS_VERSION 0x6801 /* Joystick emulation input device */ +#define HDAPS_INPUT_RAW_VERSION 0x4801 /* Raw accelerometer input device */ + +/* Axis orientation. */ +/* The unnatural bit-representation of inversions is for backward + * compatibility with the"invert=1" module parameter. */ +#define HDAPS_ORIENT_INVERT_XY 0x01 /* Invert both X and Y axes. */ +#define HDAPS_ORIENT_INVERT_X 0x02 /* Invert the X axis (uninvert if + * already inverted by INVERT_XY). */ +#define HDAPS_ORIENT_SWAP 0x04 /* Swap the axes. The swap occurs + * before inverting X or Y. */ +#define HDAPS_ORIENT_MAX 0x07 +#define HDAPS_ORIENT_UNDEFINED 0xFF /* Placeholder during initialization */ +#define HDAPS_ORIENT_INVERT_Y (HDAPS_ORIENT_INVERT_XY | HDAPS_ORIENT_INVERT_X) + +static struct timer_list hdaps_timer; static struct platform_device *pdev; -static struct input_polled_dev *hdaps_idev; -static unsigned int hdaps_invert; -static u8 km_activity; -static int rest_x; -static int rest_y; - -static DEFINE_MUTEX(hdaps_mtx); - -/* - * __get_latch - Get the value from a given port. Callers must hold hdaps_mtx. - */ -static inline u8 __get_latch(u16 port) +static struct input_dev *hdaps_idev; /* joystick-like device with fuzz */ +static struct input_dev *hdaps_idev_raw; /* raw hdaps sensor readouts */ +static unsigned int hdaps_invert = HDAPS_ORIENT_UNDEFINED; +static int needs_calibration; + +/* Configuration: */ +static int sampling_rate = 50; /* Sampling rate */ +static int oversampling_ratio = 5; /* Ratio between our sampling rate and + * EC accelerometer sampling rate */ +static int running_avg_filter_order = 2; /* EC running average filter order */ + +/* Latest state readout: */ +static int pos_x, pos_y; /* position */ +static int temperature; /* temperature */ +static int stale_readout = 1; /* last read invalid */ +static int rest_x, rest_y; /* calibrated rest position */ + +/* Last time we saw keyboard and mouse activity: */ +static u64 last_keyboard_jiffies = INITIAL_JIFFIES; +static u64 last_mouse_jiffies = INITIAL_JIFFIES; +static u64 last_update_jiffies = INITIAL_JIFFIES; + +/* input device use count */ +static int hdaps_users; +static DEFINE_MUTEX(hdaps_users_mtx); + +/* Some models require an axis transformation to the standard representation */ +static void transform_axes(int *x, int *y) { - return inb(port) & 0xff; + if (hdaps_invert & HDAPS_ORIENT_SWAP) { + int z; + z = *x; + *x = *y; + *y = z; + } + if (hdaps_invert & HDAPS_ORIENT_INVERT_XY) { + *x = -*x; + *y = -*y; + } + if (hdaps_invert & HDAPS_ORIENT_INVERT_X) + *x = -*x; } -/* - * __check_latch - Check a port latch for a given value. Returns zero if the - * port contains the given value. Callers must hold hdaps_mtx. +/** + * __hdaps_update - query current state, with locks already acquired + * @fast: if nonzero, do one quick attempt without retries. + * + * Query current accelerometer state and update global state variables. + * Also prefetches the next query. Caller must hold controller lock. */ -static inline int __check_latch(u16 port, u8 val) +static int __hdaps_update(int fast) { - if (__get_latch(port) == val) - return 0; - return -EINVAL; -} + /* Read data: */ + struct thinkpad_ec_row data; + int ret; -/* - * __wait_latch - Wait up to 100us for a port latch to get a certain value, - * returning zero if the value is obtained. Callers must hold hdaps_mtx. - */ -static int __wait_latch(u16 port, u8 val) -{ - unsigned int i; + data.mask = (1 << EC_ACCEL_IDX_READOUTS) | (1 << EC_ACCEL_IDX_KMACT) | + (3 << EC_ACCEL_IDX_YPOS1) | (3 << EC_ACCEL_IDX_XPOS1) | + (1 << EC_ACCEL_IDX_TEMP1) | (1 << EC_ACCEL_IDX_RETVAL); + if (fast) + ret = thinkpad_ec_try_read_row(&ec_accel_args, &data); + else + ret = thinkpad_ec_read_row(&ec_accel_args, &data); + thinkpad_ec_prefetch_row(&ec_accel_args); /* Prefetch even if error */ + if (ret) + return ret; - for (i = 0; i < 20; i++) { - if (!__check_latch(port, val)) - return 0; - udelay(5); + /* Check status: */ + if (data.val[EC_ACCEL_IDX_RETVAL] != 0x00) { + pr_warn("read RETVAL=0x%02x\n", + data.val[EC_ACCEL_IDX_RETVAL]); + return -EIO; + } + + if (data.val[EC_ACCEL_IDX_READOUTS] < 1) + return -EBUSY; /* no pending readout, try again later */ + + /* Parse position data: */ + pos_x = *(s16 *)(data.val+EC_ACCEL_IDX_XPOS1); + pos_y = *(s16 *)(data.val+EC_ACCEL_IDX_YPOS1); + transform_axes(&pos_x, &pos_y); + + /* Keyboard and mouse activity status is cleared as soon as it's read, + * so applications will eat each other's events. Thus we remember any + * event for KMACT_REMEMBER_PERIOD jiffies. + */ + if (data.val[EC_ACCEL_IDX_KMACT] & KEYBD_MASK) + last_keyboard_jiffies = get_jiffies_64(); + if (data.val[EC_ACCEL_IDX_KMACT] & MOUSE_MASK) + last_mouse_jiffies = get_jiffies_64(); + + temperature = data.val[EC_ACCEL_IDX_TEMP1]; + + last_update_jiffies = get_jiffies_64(); + stale_readout = 0; + if (needs_calibration) { + rest_x = pos_x; + rest_y = pos_y; + needs_calibration = 0; } - return -EIO; + return 0; } -/* - * __device_refresh - request a refresh from the accelerometer. Does not wait - * for refresh to complete. Callers must hold hdaps_mtx. +/** + * hdaps_update - acquire locks and query current state + * + * Query current accelerometer state and update global state variables. + * Also prefetches the next query. + * Retries until timeout if the accelerometer is not in ready status (common). + * Does its own locking. */ -static void __device_refresh(void) +static int hdaps_update(void) { - udelay(200); - if (inb(0x1604) != STATE_FRESH) { - outb(0x11, 0x1610); - outb(0x01, 0x161f); + u64 age = get_jiffies_64() - last_update_jiffies; + int total, ret; + + if (!stale_readout && age < (9*HZ)/(10*sampling_rate)) + return 0; /* already updated recently */ + for (total = 0; total < READ_TIMEOUT_MSECS; total += RETRY_MSECS) { + ret = thinkpad_ec_lock(); + if (ret) + return ret; + ret = __hdaps_update(0); + thinkpad_ec_unlock(); + + if (!ret) + return 0; + if (ret != -EBUSY) + break; + msleep(RETRY_MSECS); } + return ret; } -/* - * __device_refresh_sync - request a synchronous refresh from the - * accelerometer. We wait for the refresh to complete. Returns zero if - * successful and nonzero on error. Callers must hold hdaps_mtx. +/** + * hdaps_set_power - enable or disable power to the accelerometer. + * Returns zero on success and negative error code on failure. Can sleep. */ -static int __device_refresh_sync(void) +static int hdaps_set_power(int on) { - __device_refresh(); - return __wait_latch(0x1604, STATE_FRESH); + struct thinkpad_ec_row args = + { .mask = 0x0003, .val = {0x14, on?0x01:0x00} }; + struct thinkpad_ec_row data = { .mask = 0x8000 }; + int ret = thinkpad_ec_read_row(&args, &data); + if (ret) + return ret; + if (data.val[0xF] != 0x00) + return -EIO; + return 0; } -/* - * __device_complete - indicate to the accelerometer that we are done reading - * data, and then initiate an async refresh. Callers must hold hdaps_mtx. +/** + * hdaps_set_ec_config - set accelerometer parameters. + * @ec_rate: embedded controller sampling rate + * @order: embedded controller running average filter order + * (Normally we have @ec_rate = sampling_rate * oversampling_ratio.) + * Returns zero on success and negative error code on failure. Can sleep. */ -static inline void __device_complete(void) +static int hdaps_set_ec_config(int ec_rate, int order) { - inb(0x161f); - inb(0x1604); - __device_refresh(); + struct thinkpad_ec_row args = { .mask = 0x000F, + .val = {0x10, (u8)ec_rate, (u8)(ec_rate>>8), order} }; + struct thinkpad_ec_row data = { .mask = 0x8000 }; + int ret = thinkpad_ec_read_row(&args, &data); + pr_debug("setting ec_rate=%d, filter_order=%d\n", ec_rate, order); + if (ret) + return ret; + if (data.val[0xF] == 0x03) { + pr_warn("config param out of range\n"); + return -EINVAL; + } + if (data.val[0xF] == 0x06) { + pr_warn("config change already pending\n"); + return -EBUSY; + } + if (data.val[0xF] != 0x00) { + pr_warn("config change error, ret=%d\n", + data.val[0xF]); + return -EIO; + } + return 0; } -/* - * hdaps_readb_one - reads a byte from a single I/O port, placing the value in - * the given pointer. Returns zero on success or a negative error on failure. - * Can sleep. +/** + * hdaps_get_ec_config - get accelerometer parameters. + * @ec_rate: embedded controller sampling rate + * @order: embedded controller running average filter order + * Returns zero on success and negative error code on failure. Can sleep. */ -static int hdaps_readb_one(unsigned int port, u8 *val) +static int hdaps_get_ec_config(int *ec_rate, int *order) { - int ret; - - mutex_lock(&hdaps_mtx); - - /* do a sync refresh -- we need to be sure that we read fresh data */ - ret = __device_refresh_sync(); + const struct thinkpad_ec_row args = + { .mask = 0x0003, .val = {0x17, 0x82} }; + struct thinkpad_ec_row data = { .mask = 0x801F }; + int ret = thinkpad_ec_read_row(&args, &data); if (ret) - goto out; - - *val = inb(port); - __device_complete(); - -out: - mutex_unlock(&hdaps_mtx); - return ret; + return ret; + if (data.val[0xF] != 0x00) + return -EIO; + if (!(data.val[0x1] & 0x01)) + return -ENXIO; /* accelerometer polling not enabled */ + if (data.val[0x1] & 0x02) + return -EBUSY; /* config change in progress, retry later */ + *ec_rate = data.val[0x2] | ((int)(data.val[0x3]) << 8); + *order = data.val[0x4]; + return 0; } -/* __hdaps_read_pair - internal lockless helper for hdaps_read_pair(). */ -static int __hdaps_read_pair(unsigned int port1, unsigned int port2, - int *x, int *y) +/** + * hdaps_get_ec_mode - get EC accelerometer mode + * Returns zero on success and negative error code on failure. Can sleep. + */ +static int hdaps_get_ec_mode(u8 *mode) { - /* do a sync refresh -- we need to be sure that we read fresh data */ - if (__device_refresh_sync()) + const struct thinkpad_ec_row args = + { .mask = 0x0001, .val = {0x13} }; + struct thinkpad_ec_row data = { .mask = 0x8002 }; + int ret = thinkpad_ec_read_row(&args, &data); + if (ret) + return ret; + if (data.val[0xF] != 0x00) { + pr_warn("accelerometer not implemented (0x%02x)\n", + data.val[0xF]); return -EIO; - - *y = inw(port2); - *x = inw(port1); - km_activity = inb(HDAPS_PORT_KMACT); - __device_complete(); - - /* hdaps_invert is a bitvector to negate the axes */ - if (hdaps_invert & HDAPS_X_AXIS) - *x = -*x; - if (hdaps_invert & HDAPS_Y_AXIS) - *y = -*y; - + } + *mode = data.val[0x1]; return 0; } -/* - * hdaps_read_pair - reads the values from a pair of ports, placing the values - * in the given pointers. Returns zero on success. Can sleep. +/** + * hdaps_check_ec - checks something about the EC. + * Follows the clean-room spec for HDAPS; we don't know what it means. + * Returns zero on success and negative error code on failure. Can sleep. */ -static int hdaps_read_pair(unsigned int port1, unsigned int port2, - int *val1, int *val2) +static int hdaps_check_ec(void) { - int ret; - - mutex_lock(&hdaps_mtx); - ret = __hdaps_read_pair(port1, port2, val1, val2); - mutex_unlock(&hdaps_mtx); - - return ret; + const struct thinkpad_ec_row args = + { .mask = 0x0003, .val = {0x17, 0x81} }; + struct thinkpad_ec_row data = { .mask = 0x800E }; + int ret = thinkpad_ec_read_row(&args, &data); + if (ret) + return ret; + if (!((data.val[0x1] == 0x00 && data.val[0x2] == 0x60) || /* cleanroom spec */ + (data.val[0x1] == 0x01 && data.val[0x2] == 0x00)) || /* seen on T61 */ + data.val[0x3] != 0x00 || data.val[0xF] != 0x00) { + pr_warn("hdaps_check_ec: bad response (0x%x,0x%x,0x%x,0x%x)\n", + data.val[0x1], data.val[0x2], + data.val[0x3], data.val[0xF]); + return -EIO; + } + return 0; } -/* - * hdaps_device_init - initialize the accelerometer. Returns zero on success - * and negative error code on failure. Can sleep. +/** + * hdaps_device_init - initialize the accelerometer. + * + * Call several embedded controller functions to test and initialize the + * accelerometer. + * Returns zero on success and negative error code on failure. Can sleep. */ +#define FAILED_INIT(msg) pr_err("init failed at: %s\n", msg) static int hdaps_device_init(void) { - int total, ret = -ENXIO; + int ret; + u8 mode; - mutex_lock(&hdaps_mtx); + ret = thinkpad_ec_lock(); + if (ret) + return ret; - outb(0x13, 0x1610); - outb(0x01, 0x161f); - if (__wait_latch(0x161f, 0x00)) - goto out; + if (hdaps_get_ec_mode(&mode)) + { FAILED_INIT("hdaps_get_ec_mode failed"); goto bad; } - /* - * Most ThinkPads return 0x01. - * - * Others--namely the R50p, T41p, and T42p--return 0x03. These laptops - * have "inverted" axises. - * - * The 0x02 value occurs when the chip has been previously initialized. - */ - if (__check_latch(0x1611, 0x03) && - __check_latch(0x1611, 0x02) && - __check_latch(0x1611, 0x01)) - goto out; - - printk(KERN_DEBUG "hdaps: initial latch check good (0x%02x)\n", - __get_latch(0x1611)); + pr_debug("initial mode latch is 0x%02x\n", mode); + if (mode == 0x00) + { FAILED_INIT("accelerometer not available"); goto bad; } - outb(0x17, 0x1610); - outb(0x81, 0x1611); - outb(0x01, 0x161f); - if (__wait_latch(0x161f, 0x00)) - goto out; - if (__wait_latch(0x1611, 0x00)) - goto out; - if (__wait_latch(0x1612, 0x60)) - goto out; - if (__wait_latch(0x1613, 0x00)) - goto out; - outb(0x14, 0x1610); - outb(0x01, 0x1611); - outb(0x01, 0x161f); - if (__wait_latch(0x161f, 0x00)) - goto out; - outb(0x10, 0x1610); - outb(0xc8, 0x1611); - outb(0x00, 0x1612); - outb(0x02, 0x1613); - outb(0x01, 0x161f); - if (__wait_latch(0x161f, 0x00)) - goto out; - if (__device_refresh_sync()) - goto out; - if (__wait_latch(0x1611, 0x00)) - goto out; + if (hdaps_check_ec()) + { FAILED_INIT("hdaps_check_ec failed"); goto bad; } - /* we have done our dance, now let's wait for the applause */ - for (total = INIT_TIMEOUT_MSECS; total > 0; total -= INIT_WAIT_MSECS) { - int x, y; + if (hdaps_set_power(1)) + { FAILED_INIT("hdaps_set_power failed"); goto bad; } - /* a read of the device helps push it into action */ - __hdaps_read_pair(HDAPS_PORT_XPOS, HDAPS_PORT_YPOS, &x, &y); - if (!__wait_latch(0x1611, 0x02)) { - ret = 0; - break; - } + if (hdaps_set_ec_config(sampling_rate*oversampling_ratio, + running_avg_filter_order)) + { FAILED_INIT("hdaps_set_ec_config failed"); goto bad; } - msleep(INIT_WAIT_MSECS); - } + thinkpad_ec_invalidate(); + udelay(200); -out: - mutex_unlock(&hdaps_mtx); + /* Just prefetch instead of reading, to avoid ~1sec delay on load */ + ret = thinkpad_ec_prefetch_row(&ec_accel_args); + if (ret) + { FAILED_INIT("initial prefetch failed"); goto bad; } + goto good; +bad: + thinkpad_ec_invalidate(); + ret = -ENXIO; +good: + stale_readout = 1; + thinkpad_ec_unlock(); return ret; } +/** + * hdaps_device_shutdown - power off the accelerometer + * Returns nonzero on failure. Can sleep. + */ +static int hdaps_device_shutdown(void) +{ + int ret; + ret = hdaps_set_power(0); + if (ret) { + pr_warn("cannot power off\n"); + return ret; + } + ret = hdaps_set_ec_config(0, 1); + if (ret) + pr_warn("cannot stop EC sampling\n"); + return ret; +} /* Device model stuff */ @@ -306,13 +424,29 @@ static int hdaps_probe(struct platform_device *dev) } #ifdef CONFIG_PM_SLEEP +static int hdaps_suspend(struct device *dev) +{ + /* Don't do hdaps polls until resume re-initializes the sensor. */ + del_timer_sync(&hdaps_timer); + hdaps_device_shutdown(); /* ignore errors, effect is negligible */ + return 0; +} + static int hdaps_resume(struct device *dev) { - return hdaps_device_init(); + int ret = hdaps_device_init(); + if (ret) + return ret; + + mutex_lock(&hdaps_users_mtx); + if (hdaps_users) + mod_timer(&hdaps_timer, jiffies + HZ/sampling_rate); + mutex_unlock(&hdaps_users_mtx); + return 0; } #endif -static SIMPLE_DEV_PM_OPS(hdaps_pm, NULL, hdaps_resume); +static SIMPLE_DEV_PM_OPS(hdaps_pm, hdaps_suspend, hdaps_resume); static struct platform_driver hdaps_driver = { .probe = hdaps_probe, @@ -322,30 +456,47 @@ static struct platform_driver hdaps_driver = { }, }; -/* - * hdaps_calibrate - Set our "resting" values. Callers must hold hdaps_mtx. +/** + * hdaps_calibrate - set our "resting" values. + * Does its own locking. */ static void hdaps_calibrate(void) { - __hdaps_read_pair(HDAPS_PORT_XPOS, HDAPS_PORT_YPOS, &rest_x, &rest_y); + needs_calibration = 1; + hdaps_update(); + /* If that fails, the mousedev poll will take care of things later. */ } -static void hdaps_mousedev_poll(struct input_polled_dev *dev) +/* Timer handler for updating the input device. Runs in softirq context, + * so avoid lenghty or blocking operations. + */ +static void hdaps_mousedev_poll(unsigned long unused) { - struct input_dev *input_dev = dev->input; - int x, y; + int ret; - mutex_lock(&hdaps_mtx); + stale_readout = 1; - if (__hdaps_read_pair(HDAPS_PORT_XPOS, HDAPS_PORT_YPOS, &x, &y)) - goto out; + /* Cannot sleep. Try nonblockingly. If we fail, try again later. */ + if (thinkpad_ec_try_lock()) + goto keep_active; - input_report_abs(input_dev, ABS_X, x - rest_x); - input_report_abs(input_dev, ABS_Y, y - rest_y); - input_sync(input_dev); + ret = __hdaps_update(1); /* fast update, we're in softirq context */ + thinkpad_ec_unlock(); + /* Any of "successful", "not yet ready" and "not prefetched"? */ + if (ret != 0 && ret != -EBUSY && ret != -ENODATA) { + pr_err("poll failed, disabling updates\n"); + return; + } -out: - mutex_unlock(&hdaps_mtx); +keep_active: + /* Even if we failed now, pos_x,y may have been updated earlier: */ + input_report_abs(hdaps_idev, ABS_X, pos_x - rest_x); + input_report_abs(hdaps_idev, ABS_Y, pos_y - rest_y); + input_sync(hdaps_idev); + input_report_abs(hdaps_idev_raw, ABS_X, pos_x); + input_report_abs(hdaps_idev_raw, ABS_Y, pos_y); + input_sync(hdaps_idev_raw); + mod_timer(&hdaps_timer, jiffies + HZ/sampling_rate); } @@ -354,65 +505,41 @@ out: static ssize_t hdaps_position_show(struct device *dev, struct device_attribute *attr, char *buf) { - int ret, x, y; - - ret = hdaps_read_pair(HDAPS_PORT_XPOS, HDAPS_PORT_YPOS, &x, &y); - if (ret) - return ret; - - return sprintf(buf, "(%d,%d)\n", x, y); -} - -static ssize_t hdaps_variance_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - int ret, x, y; - - ret = hdaps_read_pair(HDAPS_PORT_XVAR, HDAPS_PORT_YVAR, &x, &y); + int ret = hdaps_update(); if (ret) return ret; - - return sprintf(buf, "(%d,%d)\n", x, y); + return sprintf(buf, "(%d,%d)\n", pos_x, pos_y); } static ssize_t hdaps_temp1_show(struct device *dev, struct device_attribute *attr, char *buf) { - u8 uninitialized_var(temp); - int ret; - - ret = hdaps_readb_one(HDAPS_PORT_TEMP1, &temp); - if (ret) - return ret; - - return sprintf(buf, "%u\n", temp); -} - -static ssize_t hdaps_temp2_show(struct device *dev, - struct device_attribute *attr, char *buf) -{ - u8 uninitialized_var(temp); - int ret; - - ret = hdaps_readb_one(HDAPS_PORT_TEMP2, &temp); + int ret = hdaps_update(); if (ret) return ret; - - return sprintf(buf, "%u\n", temp); + return sprintf(buf, "%d\n", temperature); } static ssize_t hdaps_keyboard_activity_show(struct device *dev, struct device_attribute *attr, char *buf) { - return sprintf(buf, "%u\n", KEYBD_ISSET(km_activity)); + int ret = hdaps_update(); + if (ret) + return ret; + return sprintf(buf, "%u\n", + get_jiffies_64() < last_keyboard_jiffies + KMACT_REMEMBER_PERIOD); } static ssize_t hdaps_mouse_activity_show(struct device *dev, struct device_attribute *attr, char *buf) { - return sprintf(buf, "%u\n", MOUSE_ISSET(km_activity)); + int ret = hdaps_update(); + if (ret) + return ret; + return sprintf(buf, "%u\n", + get_jiffies_64() < last_mouse_jiffies + KMACT_REMEMBER_PERIOD); } static ssize_t hdaps_calibrate_show(struct device *dev, @@ -425,10 +552,7 @@ static ssize_t hdaps_calibrate_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - mutex_lock(&hdaps_mtx); hdaps_calibrate(); - mutex_unlock(&hdaps_mtx); - return count; } @@ -445,7 +569,7 @@ static ssize_t hdaps_invert_store(struct device *dev, int invert; if (sscanf(buf, "%d", &invert) != 1 || - invert < 0 || invert > HDAPS_BOTH_AXES) + invert < 0 || invert > HDAPS_ORIENT_MAX) return -EINVAL; hdaps_invert = invert; @@ -454,24 +578,128 @@ static ssize_t hdaps_invert_store(struct device *dev, return count; } +static ssize_t hdaps_sampling_rate_show( + struct device *dev, struct device_attribute *attr, char *buf) +{ + return sprintf(buf, "%d\n", sampling_rate); +} + +static ssize_t hdaps_sampling_rate_store( + struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + int rate, ret; + if (sscanf(buf, "%d", &rate) != 1 || rate > HZ || rate <= 0) { + pr_warn("must have 0ident); - return 1; -} - /* hdaps_dmi_match_invert - found an inverted match. */ static int __init hdaps_dmi_match_invert(const struct dmi_system_id *id) { - hdaps_invert = (unsigned long)id->driver_data; - pr_info("inverting axis (%u) readings\n", hdaps_invert); - return hdaps_dmi_match(id); + unsigned int orient = (kernel_ulong_t) id->driver_data; + hdaps_invert = orient; + pr_info("%s detected, setting orientation %u\n", id->ident, orient); + return 1; /* stop enumeration */ } -#define HDAPS_DMI_MATCH_INVERT(vendor, model, axes) { \ +#define HDAPS_DMI_MATCH_INVERT(vendor, model, orient) { \ .ident = vendor " " model, \ .callback = hdaps_dmi_match_invert, \ - .driver_data = (void *)axes, \ + .driver_data = (void *)(orient), \ .matches = { \ DMI_MATCH(DMI_BOARD_VENDOR, vendor), \ DMI_MATCH(DMI_PRODUCT_VERSION, model) \ } \ } -#define HDAPS_DMI_MATCH_NORMAL(vendor, model) \ - HDAPS_DMI_MATCH_INVERT(vendor, model, 0) - -/* Note that HDAPS_DMI_MATCH_NORMAL("ThinkPad T42") would match - "ThinkPad T42p", so the order of the entries matters. - If your ThinkPad is not recognized, please update to latest - BIOS. This is especially the case for some R52 ThinkPads. */ -static struct dmi_system_id __initdata hdaps_whitelist[] = { - HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad R50p", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad R50"), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad R51"), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad R52"), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad R61i", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad R61", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad T41p", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad T41"), - HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad T42p", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad T42"), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad T43"), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T400", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T60", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T61p", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T61", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad X40"), - HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad X41", HDAPS_Y_AXIS), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X60", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X61s", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X61", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_NORMAL("IBM", "ThinkPad Z60m"), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad Z61m", HDAPS_BOTH_AXES), - HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad Z61p", HDAPS_BOTH_AXES), +/* List of models with abnormal axis configuration. + Note that HDAPS_DMI_MATCH_NORMAL("ThinkPad T42") would match + "ThinkPad T42p", and enumeration stops after first match, + so the order of the entries matters. */ +struct dmi_system_id __initdata hdaps_whitelist[] = { + HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad R50p", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad R60", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad T41p", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad T42p", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad X40", HDAPS_ORIENT_INVERT_Y), + HDAPS_DMI_MATCH_INVERT("IBM", "ThinkPad X41", HDAPS_ORIENT_INVERT_Y), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad R60", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad R61", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad R400", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad R500", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T60", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T61", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X60 Tablet", HDAPS_ORIENT_INVERT_Y), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X60s", HDAPS_ORIENT_INVERT_Y), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X60", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_X), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X61", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_X), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T400s", HDAPS_ORIENT_INVERT_X), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T400", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T410s", HDAPS_ORIENT_SWAP), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T410", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T500", HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad T510", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_X | HDAPS_ORIENT_INVERT_Y), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad W510", HDAPS_ORIENT_MAX), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad W520", HDAPS_ORIENT_MAX), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X200s", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X200", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_X | HDAPS_ORIENT_INVERT_Y), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X201 Tablet", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X201s", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_XY), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X201", HDAPS_ORIENT_SWAP | HDAPS_ORIENT_INVERT_X), + HDAPS_DMI_MATCH_INVERT("LENOVO", "ThinkPad X220", HDAPS_ORIENT_SWAP), { .ident = NULL } }; static int __init hdaps_init(void) { - struct input_dev *idev; int ret; - if (!dmi_check_system(hdaps_whitelist)) { - pr_warn("supported laptop not found!\n"); - ret = -ENODEV; - goto out; - } - - if (!request_region(HDAPS_LOW_PORT, HDAPS_NR_PORTS, "hdaps")) { - ret = -ENXIO; - goto out; - } + /* Determine axis orientation orientation */ + if (hdaps_invert == HDAPS_ORIENT_UNDEFINED) /* set by module param? */ + if (dmi_check_system(hdaps_whitelist) < 1) /* in whitelist? */ + hdaps_invert = 0; /* default */ + /* Init timer before platform_driver_register, in case of suspend */ + init_timer(&hdaps_timer); + hdaps_timer.function = hdaps_mousedev_poll; ret = platform_driver_register(&hdaps_driver); if (ret) - goto out_region; + goto out; pdev = platform_device_register_simple("hdaps", -1, NULL, 0); if (IS_ERR(pdev)) { @@ -571,47 +793,79 @@ static int __init hdaps_init(void) if (ret) goto out_device; - hdaps_idev = input_allocate_polled_device(); + hdaps_idev = input_allocate_device(); if (!hdaps_idev) { ret = -ENOMEM; goto out_group; } - hdaps_idev->poll = hdaps_mousedev_poll; - hdaps_idev->poll_interval = HDAPS_POLL_INTERVAL; - - /* initial calibrate for the input device */ - hdaps_calibrate(); + hdaps_idev_raw = input_allocate_device(); + if (!hdaps_idev_raw) { + ret = -ENOMEM; + goto out_idev_first; + } - /* initialize the input class */ - idev = hdaps_idev->input; - idev->name = "hdaps"; - idev->phys = "isa1600/input0"; - idev->id.bustype = BUS_ISA; - idev->dev.parent = &pdev->dev; - idev->evbit[0] = BIT_MASK(EV_ABS); - input_set_abs_params(idev, ABS_X, + /* calibration for the input device (deferred to avoid delay) */ + needs_calibration = 1; + + /* initialize the joystick-like fuzzed input device */ + hdaps_idev->name = "ThinkPad HDAPS joystick emulation"; + hdaps_idev->phys = "hdaps/input0"; + hdaps_idev->id.bustype = BUS_HOST; + hdaps_idev->id.vendor = HDAPS_INPUT_VENDOR; + hdaps_idev->id.product = HDAPS_INPUT_PRODUCT; + hdaps_idev->id.version = HDAPS_INPUT_JS_VERSION; +#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,25) + hdaps_idev->cdev.dev = &pdev->dev; +#endif + hdaps_idev->evbit[0] = BIT(EV_ABS); + hdaps_idev->open = hdaps_mousedev_open; + hdaps_idev->close = hdaps_mousedev_close; + input_set_abs_params(hdaps_idev, ABS_X, -256, 256, HDAPS_INPUT_FUZZ, HDAPS_INPUT_FLAT); - input_set_abs_params(idev, ABS_Y, + input_set_abs_params(hdaps_idev, ABS_Y, -256, 256, HDAPS_INPUT_FUZZ, HDAPS_INPUT_FLAT); - ret = input_register_polled_device(hdaps_idev); + ret = input_register_device(hdaps_idev); if (ret) goto out_idev; - pr_info("driver successfully loaded\n"); + /* initialize the raw data input device */ + hdaps_idev_raw->name = "ThinkPad HDAPS accelerometer data"; + hdaps_idev_raw->phys = "hdaps/input1"; + hdaps_idev_raw->id.bustype = BUS_HOST; + hdaps_idev_raw->id.vendor = HDAPS_INPUT_VENDOR; + hdaps_idev_raw->id.product = HDAPS_INPUT_PRODUCT; + hdaps_idev_raw->id.version = HDAPS_INPUT_RAW_VERSION; +#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,25) + hdaps_idev_raw->cdev.dev = &pdev->dev; +#endif + hdaps_idev_raw->evbit[0] = BIT(EV_ABS); + hdaps_idev_raw->open = hdaps_mousedev_open; + hdaps_idev_raw->close = hdaps_mousedev_close; + input_set_abs_params(hdaps_idev_raw, ABS_X, -32768, 32767, 0, 0); + input_set_abs_params(hdaps_idev_raw, ABS_Y, -32768, 32767, 0, 0); + + ret = input_register_device(hdaps_idev_raw); + if (ret) + goto out_idev_reg_first; + + pr_info("driver successfully loaded.\n"); return 0; +out_idev_reg_first: + input_unregister_device(hdaps_idev); out_idev: - input_free_polled_device(hdaps_idev); + input_free_device(hdaps_idev_raw); +out_idev_first: + input_free_device(hdaps_idev); out_group: sysfs_remove_group(&pdev->dev.kobj, &hdaps_attribute_group); out_device: platform_device_unregister(pdev); out_driver: platform_driver_unregister(&hdaps_driver); -out_region: - release_region(HDAPS_LOW_PORT, HDAPS_NR_PORTS); + hdaps_device_shutdown(); out: pr_warn("driver init failed (ret=%d)!\n", ret); return ret; @@ -619,12 +873,12 @@ out: static void __exit hdaps_exit(void) { - input_unregister_polled_device(hdaps_idev); - input_free_polled_device(hdaps_idev); + input_unregister_device(hdaps_idev_raw); + input_unregister_device(hdaps_idev); + hdaps_device_shutdown(); /* ignore errors, effect is negligible */ sysfs_remove_group(&pdev->dev.kobj, &hdaps_attribute_group); platform_device_unregister(pdev); platform_driver_unregister(&hdaps_driver); - release_region(HDAPS_LOW_PORT, HDAPS_NR_PORTS); pr_info("driver unloaded\n"); } @@ -632,9 +886,8 @@ static void __exit hdaps_exit(void) module_init(hdaps_init); module_exit(hdaps_exit); -module_param_named(invert, hdaps_invert, int, 0); -MODULE_PARM_DESC(invert, "invert data along each axis. 1 invert x-axis, " - "2 invert y-axis, 3 invert both axes."); +module_param_named(invert, hdaps_invert, uint, 0); +MODULE_PARM_DESC(invert, "axis orientation code"); MODULE_AUTHOR("Robert Love"); MODULE_DESCRIPTION("IBM Hard Drive Active Protection System (HDAPS) driver"); diff --git b/drivers/platform/x86/thinkpad_ec.c b/drivers/platform/x86/thinkpad_ec.c new file mode 100644 index 0000000..597614b --- /dev/null +++ b/drivers/platform/x86/thinkpad_ec.c @@ -0,0 +1,513 @@ +/* + * thinkpad_ec.c - ThinkPad embedded controller LPC3 functions + * + * The embedded controller on ThinkPad laptops has a non-standard interface, + * where LPC channel 3 of the H8S EC chip is hooked up to IO ports + * 0x1600-0x161F and implements (a special case of) the H8S LPC protocol. + * The EC LPC interface provides various system management services (currently + * known: battery information and accelerometer readouts). This driver + * provides access and mutual exclusion for the EC interface. +* + * The LPC protocol and terminology are documented here: + * "H8S/2104B Group Hardware Manual", + * http://documentation.renesas.com/eng/products/mpumcu/rej09b0300_2140bhm.pdf + * + * Copyright (C) 2006-2007 Shem Multinymous + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,26) + #include +#else + #include +#endif + +#define TP_VERSION "0.42" + +MODULE_AUTHOR("Shem Multinymous"); +MODULE_DESCRIPTION("ThinkPad embedded controller hardware access"); +MODULE_VERSION(TP_VERSION); +MODULE_LICENSE("GPL"); + +/* IO ports used by embedded controller LPC channel 3: */ +#define TPC_BASE_PORT 0x1600 +#define TPC_NUM_PORTS 0x20 +#define TPC_STR3_PORT 0x1604 /* Reads H8S EC register STR3 */ +#define TPC_TWR0_PORT 0x1610 /* Mapped to H8S EC register TWR0MW/SW */ +#define TPC_TWR15_PORT 0x161F /* Mapped to H8S EC register TWR15. */ + /* (and port TPC_TWR0_PORT+i is mapped to H8S reg TWRi for 00x%02x", \ + msg, args->val[0x0], args->val[0xF], code) + +/* State of request prefetching: */ +static u8 prefetch_arg0, prefetch_argF; /* Args of last prefetch */ +static u64 prefetch_jiffies; /* time of prefetch, or: */ +#define TPC_PREFETCH_NONE INITIAL_JIFFIES /* No prefetch */ +#define TPC_PREFETCH_JUNK (INITIAL_JIFFIES+1) /* Ignore prefetch */ + +/* Locking: */ +#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,37) +static DECLARE_MUTEX(thinkpad_ec_mutex); +#else +static DEFINE_SEMAPHORE(thinkpad_ec_mutex); +#endif + +/* Kludge in case the ACPI DSDT reserves the ports we need. */ +static bool force_io; /* Willing to do IO to ports we couldn't reserve? */ +static int reserved_io; /* Successfully reserved the ports? */ +module_param_named(force_io, force_io, bool, 0600); +MODULE_PARM_DESC(force_io, "Force IO even if region already reserved (0=off, 1=on)"); + +/** + * thinkpad_ec_lock - get lock on the ThinkPad EC + * + * Get exclusive lock for accesing the ThinkPad embedded controller LPC3 + * interface. Returns 0 iff lock acquired. + */ +int thinkpad_ec_lock(void) +{ + int ret; + ret = down_interruptible(&thinkpad_ec_mutex); + return ret; +} +EXPORT_SYMBOL_GPL(thinkpad_ec_lock); + +/** + * thinkpad_ec_try_lock - try getting lock on the ThinkPad EC + * + * Try getting an exclusive lock for accesing the ThinkPad embedded + * controller LPC3. Returns immediately if lock is not available; neither + * blocks nor sleeps. Returns 0 iff lock acquired . + */ +int thinkpad_ec_try_lock(void) +{ + return down_trylock(&thinkpad_ec_mutex); +} +EXPORT_SYMBOL_GPL(thinkpad_ec_try_lock); + +/** + * thinkpad_ec_unlock - release lock on ThinkPad EC + * + * Release a previously acquired exclusive lock on the ThinkPad ebmedded + * controller LPC3 interface. + */ +void thinkpad_ec_unlock(void) +{ + up(&thinkpad_ec_mutex); +} +EXPORT_SYMBOL_GPL(thinkpad_ec_unlock); + +/** + * thinkpad_ec_request_row - tell embedded controller to prepare a row + * @args Input register arguments + * + * Requests a data row by writing to H8S LPC registers TRW0 through TWR15 (or + * a subset thereof) following the protocol prescribed by the "H8S/2104B Group + * Hardware Manual". Does sanity checks via status register STR3. + */ +static int thinkpad_ec_request_row(const struct thinkpad_ec_row *args) +{ + u8 str3; + int i; + + /* EC protocol requires write to TWR0 (function code): */ + if (!(args->mask & 0x0001)) { + printk(KERN_ERR MSG_FMT("bad args->mask=0x%02x", args->mask)); + return -EINVAL; + } + + /* Check initial STR3 status: */ + str3 = inb(TPC_STR3_PORT) & H8S_STR3_MASK; + if (str3 & H8S_STR3_OBF3B) { /* data already pending */ + inb(TPC_TWR15_PORT); /* marks end of previous transaction */ + if (prefetch_jiffies == TPC_PREFETCH_NONE) + printk(KERN_WARNING REQ_FMT( + "EC has result from unrequested transaction", + str3)); + return -EBUSY; /* EC will be ready in a few usecs */ + } else if (str3 == H8S_STR3_SWMF) { /* busy with previous request */ + if (prefetch_jiffies == TPC_PREFETCH_NONE) + printk(KERN_WARNING REQ_FMT( + "EC is busy with unrequested transaction", + str3)); + return -EBUSY; /* data will be pending in a few usecs */ + } else if (str3 != 0x00) { /* unexpected status? */ + printk(KERN_WARNING REQ_FMT("unexpected initial STR3", str3)); + return -EIO; + } + + /* Send TWR0MW: */ + outb(args->val[0], TPC_TWR0_PORT); + str3 = inb(TPC_STR3_PORT) & H8S_STR3_MASK; + if (str3 != H8S_STR3_MWMF) { /* not accepted? */ + printk(KERN_WARNING REQ_FMT("arg0 rejected", str3)); + return -EIO; + } + + /* Send TWR1 through TWR14: */ + for (i = 1; i < TP_CONTROLLER_ROW_LEN-1; i++) + if ((args->mask>>i)&1) + outb(args->val[i], TPC_TWR0_PORT+i); + + /* Send TWR15 (default to 0x01). This marks end of command. */ + outb((args->mask & 0x8000) ? args->val[0xF] : 0x01, TPC_TWR15_PORT); + + /* Wait until EC starts writing its reply (~60ns on average). + * Releasing locks before this happens may cause an EC hang + * due to firmware bug! + */ + for (i = 0; i < TPC_REQUEST_RETRIES; i++) { + str3 = inb(TPC_STR3_PORT) & H8S_STR3_MASK; + if (str3 & H8S_STR3_SWMF) /* EC started replying */ + return 0; + else if (!(str3 & ~(H8S_STR3_IBF3B|H8S_STR3_MWMF))) + /* Normal progress (the EC hasn't seen the request + * yet, or is processing it). Wait it out. */ + ndelay(TPC_REQUEST_NDELAY); + else { /* weird EC status */ + printk(KERN_WARNING + REQ_FMT("bad end STR3", str3)); + return -EIO; + } + } + printk(KERN_WARNING REQ_FMT("EC is mysteriously silent", str3)); + return -EIO; +} + +/** + * thinkpad_ec_read_data - read pre-requested row-data from EC + * @args Input register arguments of pre-requested rows + * @data Output register values + * + * Reads current row data from the controller, assuming it's already + * requested. Follows the H8S spec for register access and status checks. + */ +static int thinkpad_ec_read_data(const struct thinkpad_ec_row *args, + struct thinkpad_ec_row *data) +{ + int i; + u8 str3 = inb(TPC_STR3_PORT) & H8S_STR3_MASK; + /* Once we make a request, STR3 assumes the sequence of values listed + * in the following 'if' as it reads the request and writes its data. + * It takes about a few dozen nanosecs total, with very high variance. + */ + if (str3 == (H8S_STR3_IBF3B|H8S_STR3_MWMF) || + str3 == 0x00 || /* the 0x00 is indistinguishable from idle EC! */ + str3 == H8S_STR3_SWMF) + return -EBUSY; /* not ready yet */ + /* Finally, the EC signals output buffer full: */ + if (str3 != (H8S_STR3_OBF3B|H8S_STR3_SWMF)) { + printk(KERN_WARNING + REQ_FMT("bad initial STR3", str3)); + return -EIO; + } + + /* Read first byte (signals start of read transactions): */ + data->val[0] = inb(TPC_TWR0_PORT); + /* Optionally read 14 more bytes: */ + for (i = 1; i < TP_CONTROLLER_ROW_LEN-1; i++) + if ((data->mask >> i)&1) + data->val[i] = inb(TPC_TWR0_PORT+i); + /* Read last byte from 0x161F (signals end of read transaction): */ + data->val[0xF] = inb(TPC_TWR15_PORT); + + /* Readout still pending? */ + str3 = inb(TPC_STR3_PORT) & H8S_STR3_MASK; + if (str3 & H8S_STR3_OBF3B) + printk(KERN_WARNING + REQ_FMT("OBF3B=1 after read", str3)); + /* If port 0x161F returns 0x80 too often, the EC may lock up. Warn: */ + if (data->val[0xF] == 0x80) + printk(KERN_WARNING + REQ_FMT("0x161F reports error", data->val[0xF])); + return 0; +} + +/** + * thinkpad_ec_is_row_fetched - is the given row currently prefetched? + * + * To keep things simple we compare only the first and last args; + * this suffices for all known cases. + */ +static int thinkpad_ec_is_row_fetched(const struct thinkpad_ec_row *args) +{ + return (prefetch_jiffies != TPC_PREFETCH_NONE) && + (prefetch_jiffies != TPC_PREFETCH_JUNK) && + (prefetch_arg0 == args->val[0]) && + (prefetch_argF == args->val[0xF]) && + (get_jiffies_64() < prefetch_jiffies + TPC_PREFETCH_TIMEOUT); +} + +/** + * thinkpad_ec_read_row - request and read data from ThinkPad EC + * @args Input register arguments + * @data Output register values + * + * Read a data row from the ThinkPad embedded controller LPC3 interface. + * Does fetching and retrying if needed. The row is specified by an + * array of 16 bytes, some of which may be undefined (but the first is + * mandatory). These bytes are given in @args->val[], where @args->val[i] is + * used iff (@args->mask>>i)&1). The resulting row data is stored in + * @data->val[], but is only guaranteed to be valid for indices corresponding + * to set bit in @data->mask. That is, if @data->mask&(1<val[i] is undefined. + * + * Returns -EBUSY on transient error and -EIO on abnormal condition. + * Caller must hold controller lock. + */ +int thinkpad_ec_read_row(const struct thinkpad_ec_row *args, + struct thinkpad_ec_row *data) +{ + int retries, ret; + + if (thinkpad_ec_is_row_fetched(args)) + goto read_row; /* already requested */ + + /* Request the row */ + for (retries = 0; retries < TPC_READ_RETRIES; ++retries) { + ret = thinkpad_ec_request_row(args); + if (!ret) + goto read_row; + if (ret != -EBUSY) + break; + ndelay(TPC_READ_NDELAY); + } + printk(KERN_ERR REQ_FMT("failed requesting row", ret)); + goto out; + +read_row: + /* Read the row's data */ + for (retries = 0; retries < TPC_READ_RETRIES; ++retries) { + ret = thinkpad_ec_read_data(args, data); + if (!ret) + goto out; + if (ret != -EBUSY) + break; + ndelay(TPC_READ_NDELAY); + } + + printk(KERN_ERR REQ_FMT("failed waiting for data", ret)); + +out: + prefetch_jiffies = TPC_PREFETCH_JUNK; + return ret; +} +EXPORT_SYMBOL_GPL(thinkpad_ec_read_row); + +/** + * thinkpad_ec_try_read_row - try reading prefetched data from ThinkPad EC + * @args Input register arguments + * @data Output register values + * + * Try reading a data row from the ThinkPad embedded controller LPC3 + * interface, if this raw was recently prefetched using + * thinkpad_ec_prefetch_row(). Does not fetch, retry or block. + * The parameters have the same meaning as in thinkpad_ec_read_row(). + * + * Returns -EBUSY is data not ready and -ENODATA if row not prefetched. + * Caller must hold controller lock. + */ +int thinkpad_ec_try_read_row(const struct thinkpad_ec_row *args, + struct thinkpad_ec_row *data) +{ + int ret; + if (!thinkpad_ec_is_row_fetched(args)) { + ret = -ENODATA; + } else { + ret = thinkpad_ec_read_data(args, data); + if (!ret) + prefetch_jiffies = TPC_PREFETCH_NONE; /* eaten up */ + } + return ret; +} +EXPORT_SYMBOL_GPL(thinkpad_ec_try_read_row); + +/** + * thinkpad_ec_prefetch_row - prefetch data from ThinkPad EC + * @args Input register arguments + * + * Prefetch a data row from the ThinkPad embedded controller LCP3 + * interface. A subsequent call to thinkpad_ec_read_row() with the + * same arguments will be faster, and a subsequent call to + * thinkpad_ec_try_read_row() stands a good chance of succeeding if + * done neither too soon nor too late. See + * thinkpad_ec_read_row() for the meaning of @args. + * + * Returns -EBUSY on transient error and -EIO on abnormal condition. + * Caller must hold controller lock. + */ +int thinkpad_ec_prefetch_row(const struct thinkpad_ec_row *args) +{ + int ret; + ret = thinkpad_ec_request_row(args); + if (ret) { + prefetch_jiffies = TPC_PREFETCH_JUNK; + } else { + prefetch_jiffies = get_jiffies_64(); + prefetch_arg0 = args->val[0x0]; + prefetch_argF = args->val[0xF]; + } + return ret; +} +EXPORT_SYMBOL_GPL(thinkpad_ec_prefetch_row); + +/** + * thinkpad_ec_invalidate - invalidate prefetched ThinkPad EC data + * + * Invalidate the data prefetched via thinkpad_ec_prefetch_row() from the + * ThinkPad embedded controller LPC3 interface. + * Must be called before unlocking by any code that accesses the controller + * ports directly. + */ +void thinkpad_ec_invalidate(void) +{ + prefetch_jiffies = TPC_PREFETCH_JUNK; +} +EXPORT_SYMBOL_GPL(thinkpad_ec_invalidate); + + +/*** Checking for EC hardware ***/ + +/** + * thinkpad_ec_test - verify the EC is present and follows protocol + * + * Ensure the EC LPC3 channel really works on this machine by making + * an EC request and seeing if the EC follows the documented H8S protocol. + * The requested row just reads battery status, so it should be harmless to + * access it (on a correct EC). + * This test writes to IO ports, so execute only after checking DMI. + */ +static int __init thinkpad_ec_test(void) +{ + int ret; + const struct thinkpad_ec_row args = /* battery 0 basic status */ + { .mask = 0x8001, .val = {0x01,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0x00} }; + struct thinkpad_ec_row data = { .mask = 0x0000 }; + ret = thinkpad_ec_lock(); + if (ret) + return ret; + ret = thinkpad_ec_read_row(&args, &data); + thinkpad_ec_unlock(); + return ret; +} + +/* Search all DMI device names of a given type for a substring */ +static int __init dmi_find_substring(int type, const char *substr) +{ + const struct dmi_device *dev = NULL; + while ((dev = dmi_find_device(type, NULL, dev))) { + if (strstr(dev->name, substr)) + return 1; + } + return 0; +} + +#define TP_DMI_MATCH(vendor,model) { \ + .ident = vendor " " model, \ + .matches = { \ + DMI_MATCH(DMI_BOARD_VENDOR, vendor), \ + DMI_MATCH(DMI_PRODUCT_VERSION, model) \ + } \ +} + +/* Check DMI for existence of ThinkPad embedded controller */ +static int __init check_dmi_for_ec(void) +{ + /* A few old models that have a good EC but don't report it in DMI */ + struct dmi_system_id tp_whitelist[] = { + TP_DMI_MATCH("IBM", "ThinkPad A30"), + TP_DMI_MATCH("IBM", "ThinkPad T23"), + TP_DMI_MATCH("IBM", "ThinkPad X24"), + TP_DMI_MATCH("LENOVO", "ThinkPad"), + { .ident = NULL } + }; + return dmi_find_substring(DMI_DEV_TYPE_OEM_STRING, + "IBM ThinkPad Embedded Controller") || + dmi_check_system(tp_whitelist); +} + +/*** Init and cleanup ***/ + +static int __init thinkpad_ec_init(void) +{ + if (!check_dmi_for_ec()) { + printk(KERN_WARNING + "thinkpad_ec: no ThinkPad embedded controller!\n"); + return -ENODEV; + } + + if (request_region(TPC_BASE_PORT, TPC_NUM_PORTS, "thinkpad_ec")) { + reserved_io = 1; + } else { + printk(KERN_ERR "thinkpad_ec: cannot claim IO ports %#x-%#x... ", + TPC_BASE_PORT, + TPC_BASE_PORT + TPC_NUM_PORTS - 1); + if (force_io) { + printk("forcing use of unreserved IO ports.\n"); + } else { + printk("consider using force_io=1.\n"); + return -ENXIO; + } + } + prefetch_jiffies = TPC_PREFETCH_JUNK; + if (thinkpad_ec_test()) { + printk(KERN_ERR "thinkpad_ec: initial ec test failed\n"); + if (reserved_io) + release_region(TPC_BASE_PORT, TPC_NUM_PORTS); + return -ENXIO; + } + printk(KERN_INFO "thinkpad_ec: thinkpad_ec " TP_VERSION " loaded.\n"); + return 0; +} + +static void __exit thinkpad_ec_exit(void) +{ + if (reserved_io) + release_region(TPC_BASE_PORT, TPC_NUM_PORTS); + printk(KERN_INFO "thinkpad_ec: unloaded.\n"); +} + +module_init(thinkpad_ec_init); +module_exit(thinkpad_ec_exit); diff --git b/drivers/platform/x86/tp_smapi.c b/drivers/platform/x86/tp_smapi.c new file mode 100644 index 0000000..209cb64 --- /dev/null +++ b/drivers/platform/x86/tp_smapi.c @@ -0,0 +1,1493 @@ +/* + * tp_smapi.c - ThinkPad SMAPI support + * + * This driver exposes some features of the System Management Application + * Program Interface (SMAPI) BIOS found on ThinkPad laptops. It works on + * models in which the SMAPI BIOS runs in SMM and is invoked by writing + * to the APM control port 0xB2. + * It also exposes battery status information, obtained from the ThinkPad + * embedded controller (via the thinkpad_ec module). + * Ancient ThinkPad models use a different interface, supported by the + * "thinkpad" module from "tpctl". + * + * Many of the battery status values obtained from the EC simply mirror + * values provided by the battery's Smart Battery System (SBS) interface, so + * their meaning is defined by the Smart Battery Data Specification (see + * http://sbs-forum.org/specs/sbdat110.pdf). References to this SBS spec + * are given in the code where relevant. + * + * Copyright (C) 2006 Shem Multinymous . + * SMAPI access code based on the mwave driver by Mike Sullivan. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include +#include +#include +#include +#include +#include /* CMOS defines */ +#include +#include +#include +#include +#include +#include + +#define TP_VERSION "0.42" +#define TP_DESC "ThinkPad SMAPI Support" +#define TP_DIR "smapi" + +MODULE_AUTHOR("Shem Multinymous"); +MODULE_DESCRIPTION(TP_DESC); +MODULE_VERSION(TP_VERSION); +MODULE_LICENSE("GPL"); + +static struct platform_device *pdev; + +static int tp_debug; +module_param_named(debug, tp_debug, int, 0600); +MODULE_PARM_DESC(debug, "Debug level (0=off, 1=on)"); + +/* A few macros for printk()ing: */ +#define TPRINTK(level, fmt, args...) \ + dev_printk(level, &(pdev->dev), "%s: " fmt "\n", __func__, ## args) +#define DPRINTK(fmt, args...) \ + do { if (tp_debug) TPRINTK(KERN_DEBUG, fmt, ## args); } while (0) + +/********************************************************************* + * SMAPI interface + */ + +/* SMAPI functions (register BX when making the SMM call). */ +#define SMAPI_GET_INHIBIT_CHARGE 0x2114 +#define SMAPI_SET_INHIBIT_CHARGE 0x2115 +#define SMAPI_GET_THRESH_START 0x2116 +#define SMAPI_SET_THRESH_START 0x2117 +#define SMAPI_GET_FORCE_DISCHARGE 0x2118 +#define SMAPI_SET_FORCE_DISCHARGE 0x2119 +#define SMAPI_GET_THRESH_STOP 0x211a +#define SMAPI_SET_THRESH_STOP 0x211b + +/* SMAPI error codes (see ThinkPad 770 Technical Reference Manual p.83 at + http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=PFAN-3TUQQD */ +#define SMAPI_RETCODE_EOF 0xff +static struct { u8 rc; char *msg; int ret; } smapi_retcode[] = +{ + {0x00, "OK", 0}, + {0x53, "SMAPI function is not available", -ENXIO}, + {0x81, "Invalid parameter", -EINVAL}, + {0x86, "Function is not supported by SMAPI BIOS", -EOPNOTSUPP}, + {0x90, "System error", -EIO}, + {0x91, "System is invalid", -EIO}, + {0x92, "System is busy, -EBUSY"}, + {0xa0, "Device error (disk read error)", -EIO}, + {0xa1, "Device is busy", -EBUSY}, + {0xa2, "Device is not attached", -ENXIO}, + {0xa3, "Device is disbled", -EIO}, + {0xa4, "Request parameter is out of range", -EINVAL}, + {0xa5, "Request parameter is not accepted", -EINVAL}, + {0xa6, "Transient error", -EBUSY}, /* ? */ + {SMAPI_RETCODE_EOF, "Unknown error code", -EIO} +}; + + +#define SMAPI_MAX_RETRIES 10 +#define SMAPI_PORT2 0x4F /* fixed port, meaning unclear */ +static unsigned short smapi_port; /* APM control port, normally 0xB2 */ + +#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,37) +static DECLARE_MUTEX(smapi_mutex); +#else +static DEFINE_SEMAPHORE(smapi_mutex); +#endif + +/** + * find_smapi_port - read SMAPI port from NVRAM + */ +static int __init find_smapi_port(void) +{ + u16 smapi_id = 0; + unsigned short port = 0; + unsigned long flags; + + spin_lock_irqsave(&rtc_lock, flags); + smapi_id = CMOS_READ(0x7C); + smapi_id |= (CMOS_READ(0x7D) << 8); + spin_unlock_irqrestore(&rtc_lock, flags); + + if (smapi_id != 0x5349) { + printk(KERN_ERR "SMAPI not supported (ID=0x%x)\n", smapi_id); + return -ENXIO; + } + spin_lock_irqsave(&rtc_lock, flags); + port = CMOS_READ(0x7E); + port |= (CMOS_READ(0x7F) << 8); + spin_unlock_irqrestore(&rtc_lock, flags); + if (port == 0) { + printk(KERN_ERR "unable to read SMAPI port number\n"); + return -ENXIO; + } + return port; +} + +/** + * smapi_request - make a SMAPI call + * @inEBX, @inECX, @inEDI, @inESI: input registers + * @outEBX, @outECX, @outEDX, @outEDI, @outESI: outputs registers + * @msg: textual error message + * Invokes the SMAPI SMBIOS with the given input and outpu args. + * All outputs are optional (can be %NULL). + * Returns 0 when successful, and a negative errno constant + * (see smapi_retcode above) upon failure. + */ +static int smapi_request(u32 inEBX, u32 inECX, + u32 inEDI, u32 inESI, + u32 *outEBX, u32 *outECX, u32 *outEDX, + u32 *outEDI, u32 *outESI, const char **msg) +{ + int ret = 0; + int i; + int retries; + u8 rc; + /* Must use local vars for output regs, due to reg pressure. */ + u32 tmpEAX, tmpEBX, tmpECX, tmpEDX, tmpEDI, tmpESI; + + for (retries = 0; retries < SMAPI_MAX_RETRIES; ++retries) { + DPRINTK("req_in: BX=%x CX=%x DI=%x SI=%x", + inEBX, inECX, inEDI, inESI); + + /* SMAPI's SMBIOS call and thinkpad_ec end up using use + * different interfaces to the same chip, so play it safe. */ + ret = thinkpad_ec_lock(); + if (ret) + return ret; + + __asm__ __volatile__( + "movl $0x00005380,%%eax\n\t" + "movl %6,%%ebx\n\t" + "movl %7,%%ecx\n\t" + "movl %8,%%edi\n\t" + "movl %9,%%esi\n\t" + "xorl %%edx,%%edx\n\t" + "movw %10,%%dx\n\t" + "out %%al,%%dx\n\t" /* trigger SMI to SMBIOS */ + "out %%al,$0x4F\n\t" + "movl %%eax,%0\n\t" + "movl %%ebx,%1\n\t" + "movl %%ecx,%2\n\t" + "movl %%edx,%3\n\t" + "movl %%edi,%4\n\t" + "movl %%esi,%5\n\t" + :"=m"(tmpEAX), + "=m"(tmpEBX), + "=m"(tmpECX), + "=m"(tmpEDX), + "=m"(tmpEDI), + "=m"(tmpESI) + :"m"(inEBX), "m"(inECX), "m"(inEDI), "m"(inESI), + "m"((u16)smapi_port) + :"%eax", "%ebx", "%ecx", "%edx", "%edi", + "%esi"); + + thinkpad_ec_invalidate(); + thinkpad_ec_unlock(); + + /* Don't let the next SMAPI access happen too quickly, + * may case problems. (We're hold smapi_mutex). */ + msleep(50); + + if (outEBX) *outEBX = tmpEBX; + if (outECX) *outECX = tmpECX; + if (outEDX) *outEDX = tmpEDX; + if (outESI) *outESI = tmpESI; + if (outEDI) *outEDI = tmpEDI; + + /* Look up error code */ + rc = (tmpEAX>>8)&0xFF; + for (i = 0; smapi_retcode[i].rc != SMAPI_RETCODE_EOF && + smapi_retcode[i].rc != rc; ++i) {} + ret = smapi_retcode[i].ret; + if (msg) + *msg = smapi_retcode[i].msg; + + DPRINTK("req_out: AX=%x BX=%x CX=%x DX=%x DI=%x SI=%x r=%d", + tmpEAX, tmpEBX, tmpECX, tmpEDX, tmpEDI, tmpESI, ret); + if (ret) + TPRINTK(KERN_NOTICE, "SMAPI error: %s (func=%x)", + smapi_retcode[i].msg, inEBX); + + if (ret != -EBUSY) + return ret; + } + return ret; +} + +/* Convenience wrapper: discard output arguments */ +static int smapi_write(u32 inEBX, u32 inECX, + u32 inEDI, u32 inESI, const char **msg) +{ + return smapi_request(inEBX, inECX, inEDI, inESI, + NULL, NULL, NULL, NULL, NULL, msg); +} + + +/********************************************************************* + * Specific SMAPI services + * All of these functions return 0 upon success, and a negative errno + * constant (see smapi_retcode) on failure. + */ + +enum thresh_type { + THRESH_STOP = 0, /* the code assumes this is 0 for brevity */ + THRESH_START +}; +#define THRESH_NAME(which) ((which == THRESH_START) ? "start" : "stop") + +/** + * __get_real_thresh - read battery charge start/stop threshold from SMAPI + * @bat: battery number (0 or 1) + * @which: THRESH_START or THRESH_STOP + * @thresh: 1..99, 0=default 1..99, 0=default (pass this as-is to SMAPI) + * @outEDI: some additional state that needs to be preserved, meaning unknown + * @outESI: some additional state that needs to be preserved, meaning unknown + */ +static int __get_real_thresh(int bat, enum thresh_type which, int *thresh, + u32 *outEDI, u32 *outESI) +{ + u32 ebx = (which == THRESH_START) ? SMAPI_GET_THRESH_START + : SMAPI_GET_THRESH_STOP; + u32 ecx = (bat+1)<<8; + const char *msg; + int ret = smapi_request(ebx, ecx, 0, 0, NULL, + &ecx, NULL, outEDI, outESI, &msg); + if (ret) { + TPRINTK(KERN_NOTICE, "cannot get %s_thresh of bat=%d: %s", + THRESH_NAME(which), bat, msg); + return ret; + } + if (!(ecx&0x00000100)) { + TPRINTK(KERN_NOTICE, "cannot get %s_thresh of bat=%d: ecx=0%x", + THRESH_NAME(which), bat, ecx); + return -EIO; + } + if (thresh) + *thresh = ecx&0xFF; + return 0; +} + +/** + * get_real_thresh - read battery charge start/stop threshold from SMAPI + * @bat: battery number (0 or 1) + * @which: THRESH_START or THRESH_STOP + * @thresh: 1..99, 0=default (passes as-is to SMAPI) + */ +static int get_real_thresh(int bat, enum thresh_type which, int *thresh) +{ + return __get_real_thresh(bat, which, thresh, NULL, NULL); +} + +/** + * set_real_thresh - write battery start/top charge threshold to SMAPI + * @bat: battery number (0 or 1) + * @which: THRESH_START or THRESH_STOP + * @thresh: 1..99, 0=default (passes as-is to SMAPI) + */ +static int set_real_thresh(int bat, enum thresh_type which, int thresh) +{ + u32 ebx = (which == THRESH_START) ? SMAPI_SET_THRESH_START + : SMAPI_SET_THRESH_STOP; + u32 ecx = ((bat+1)<<8) + thresh; + u32 getDI, getSI; + const char *msg; + int ret; + + /* verify read before writing */ + ret = __get_real_thresh(bat, which, NULL, &getDI, &getSI); + if (ret) + return ret; + + ret = smapi_write(ebx, ecx, getDI, getSI, &msg); + if (ret) + TPRINTK(KERN_NOTICE, "set %s to %d for bat=%d failed: %s", + THRESH_NAME(which), thresh, bat, msg); + else + TPRINTK(KERN_INFO, "set %s to %d for bat=%d", + THRESH_NAME(which), thresh, bat); + return ret; +} + +/** + * __get_inhibit_charge_minutes - get inhibit charge period from SMAPI + * @bat: battery number (0 or 1) + * @minutes: period in minutes (1..65535 minutes, 0=disabled) + * @outECX: some additional state that needs to be preserved, meaning unknown + * Note that @minutes is the originally set value, it does not count down. + */ +static int __get_inhibit_charge_minutes(int bat, int *minutes, u32 *outECX) +{ + u32 ecx = (bat+1)<<8; + u32 esi; + const char *msg; + int ret = smapi_request(SMAPI_GET_INHIBIT_CHARGE, ecx, 0, 0, + NULL, &ecx, NULL, NULL, &esi, &msg); + if (ret) { + TPRINTK(KERN_NOTICE, "failed for bat=%d: %s", bat, msg); + return ret; + } + if (!(ecx&0x0100)) { + TPRINTK(KERN_NOTICE, "bad ecx=0x%x for bat=%d", ecx, bat); + return -EIO; + } + if (minutes) + *minutes = (ecx&0x0001)?esi:0; + if (outECX) + *outECX = ecx; + return 0; +} + +/** + * get_inhibit_charge_minutes - get inhibit charge period from SMAPI + * @bat: battery number (0 or 1) + * @minutes: period in minutes (1..65535 minutes, 0=disabled) + * Note that @minutes is the originally set value, it does not count down. + */ +static int get_inhibit_charge_minutes(int bat, int *minutes) +{ + return __get_inhibit_charge_minutes(bat, minutes, NULL); +} + +/** + * set_inhibit_charge_minutes - write inhibit charge period to SMAPI + * @bat: battery number (0 or 1) + * @minutes: period in minutes (1..65535 minutes, 0=disabled) + */ +static int set_inhibit_charge_minutes(int bat, int minutes) +{ + u32 ecx; + const char *msg; + int ret; + + /* verify read before writing */ + ret = __get_inhibit_charge_minutes(bat, NULL, &ecx); + if (ret) + return ret; + + ecx = ((bat+1)<<8) | (ecx&0x00FE) | (minutes > 0 ? 0x0001 : 0x0000); + if (minutes > 0xFFFF) + minutes = 0xFFFF; + ret = smapi_write(SMAPI_SET_INHIBIT_CHARGE, ecx, 0, minutes, &msg); + if (ret) + TPRINTK(KERN_NOTICE, + "set to %d failed for bat=%d: %s", minutes, bat, msg); + else + TPRINTK(KERN_INFO, "set to %d for bat=%d\n", minutes, bat); + return ret; +} + + +/** + * get_force_discharge - get status of forced discharging from SMAPI + * @bat: battery number (0 or 1) + * @enabled: 1 if forced discharged is enabled, 0 if not + */ +static int get_force_discharge(int bat, int *enabled) +{ + u32 ecx = (bat+1)<<8; + const char *msg; + int ret = smapi_request(SMAPI_GET_FORCE_DISCHARGE, ecx, 0, 0, + NULL, &ecx, NULL, NULL, NULL, &msg); + if (ret) { + TPRINTK(KERN_NOTICE, "failed for bat=%d: %s", bat, msg); + return ret; + } + *enabled = (!(ecx&0x00000100) && (ecx&0x00000001))?1:0; + return 0; +} + +/** + * set_force_discharge - write status of forced discharging to SMAPI + * @bat: battery number (0 or 1) + * @enabled: 1 if forced discharged is enabled, 0 if not + */ +static int set_force_discharge(int bat, int enabled) +{ + u32 ecx = (bat+1)<<8; + const char *msg; + int ret = smapi_request(SMAPI_GET_FORCE_DISCHARGE, ecx, 0, 0, + NULL, &ecx, NULL, NULL, NULL, &msg); + if (ret) { + TPRINTK(KERN_NOTICE, "get failed for bat=%d: %s", bat, msg); + return ret; + } + if (ecx&0x00000100) { + TPRINTK(KERN_NOTICE, "cannot force discharge bat=%d", bat); + return -EIO; + } + + ecx = ((bat+1)<<8) | (ecx&0x000000FA) | (enabled?0x00000001:0); + ret = smapi_write(SMAPI_SET_FORCE_DISCHARGE, ecx, 0, 0, &msg); + if (ret) + TPRINTK(KERN_NOTICE, "set to %d failed for bat=%d: %s", + enabled, bat, msg); + else + TPRINTK(KERN_INFO, "set to %d for bat=%d", enabled, bat); + return ret; +} + + +/********************************************************************* + * Wrappers to threshold-related SMAPI functions, which handle default + * thresholds and related quirks. + */ + +/* Minimum, default and minimum difference for battery charging thresholds: */ +#define MIN_THRESH_DELTA 4 /* Min delta between start and stop thresh */ +#define MIN_THRESH_START 2 +#define MAX_THRESH_START (100-MIN_THRESH_DELTA) +#define MIN_THRESH_STOP (MIN_THRESH_START + MIN_THRESH_DELTA) +#define MAX_THRESH_STOP 100 +#define DEFAULT_THRESH_START MAX_THRESH_START +#define DEFAULT_THRESH_STOP MAX_THRESH_STOP + +/* The GUI of IBM's Battery Maximizer seems to show a start threshold that + * is 1 more than the value we set/get via SMAPI. Since the threshold is + * maintained across reboot, this can be confusing. So we kludge our + * interface for interoperability: */ +#define BATMAX_FIX 1 + +/* Get charge start/stop threshold (1..100), + * substituting default values if needed and applying BATMAT_FIX. */ +static int get_thresh(int bat, enum thresh_type which, int *thresh) +{ + int ret = get_real_thresh(bat, which, thresh); + if (ret) + return ret; + if (*thresh == 0) + *thresh = (which == THRESH_START) ? DEFAULT_THRESH_START + : DEFAULT_THRESH_STOP; + else if (which == THRESH_START) + *thresh += BATMAX_FIX; + return 0; +} + + +/* Set charge start/stop threshold (1..100), + * substituting default values if needed and applying BATMAT_FIX. */ +static int set_thresh(int bat, enum thresh_type which, int thresh) +{ + if (which == THRESH_STOP && thresh == DEFAULT_THRESH_STOP) + thresh = 0; /* 100 is out of range, but default means 100 */ + if (which == THRESH_START) + thresh -= BATMAX_FIX; + return set_real_thresh(bat, which, thresh); +} + +/********************************************************************* + * ThinkPad embedded controller readout and basic functions + */ + +/** + * read_tp_ec_row - read data row from the ThinkPad embedded controller + * @arg0: EC command code + * @bat: battery number, 0 or 1 + * @j: the byte value to be used for "junk" (unused) input/outputs + * @dataval: result vector + */ +static int read_tp_ec_row(u8 arg0, int bat, u8 j, u8 *dataval) +{ + int ret; + const struct thinkpad_ec_row args = { .mask = 0xFFFF, + .val = {arg0, j,j,j,j,j,j,j,j,j,j,j,j,j,j, (u8)bat} }; + struct thinkpad_ec_row data = { .mask = 0xFFFF }; + + ret = thinkpad_ec_lock(); + if (ret) + return ret; + ret = thinkpad_ec_read_row(&args, &data); + thinkpad_ec_unlock(); + memcpy(dataval, &data.val, TP_CONTROLLER_ROW_LEN); + return ret; +} + +/** + * power_device_present - check for presence of battery or AC power + * @bat: 0 for battery 0, 1 for battery 1, otherwise AC power + * Returns 1 if present, 0 if not present, negative if error. + */ +static int power_device_present(int bat) +{ + u8 row[TP_CONTROLLER_ROW_LEN]; + u8 test; + int ret = read_tp_ec_row(1, bat, 0, row); + if (ret) + return ret; + switch (bat) { + case 0: test = 0x40; break; /* battery 0 */ + case 1: test = 0x20; break; /* battery 1 */ + default: test = 0x80; /* AC power */ + } + return (row[0] & test) ? 1 : 0; +} + +/** + * bat_has_status - check if battery can report detailed status + * @bat: 0 for battery 0, 1 for battery 1 + * Returns 1 if yes, 0 if no, negative if error. + */ +static int bat_has_status(int bat) +{ + u8 row[TP_CONTROLLER_ROW_LEN]; + int ret = read_tp_ec_row(1, bat, 0, row); + if (ret) + return ret; + if ((row[0] & (bat?0x20:0x40)) == 0) /* no battery */ + return 0; + if ((row[1] & (0x60)) == 0) /* no status */ + return 0; + return 1; +} + +/** + * get_tp_ec_bat_16 - read a 16-bit value from EC battery status data + * @arg0: first argument to EC + * @off: offset in row returned from EC + * @bat: battery (0 or 1) + * @val: the 16-bit value obtained + * Returns nonzero on error. + */ +static int get_tp_ec_bat_16(u8 arg0, int offset, int bat, u16 *val) +{ + u8 row[TP_CONTROLLER_ROW_LEN]; + int ret; + if (bat_has_status(bat) != 1) + return -ENXIO; + ret = read_tp_ec_row(arg0, bat, 0, row); + if (ret) + return ret; + *val = *(u16 *)(row+offset); + return 0; +} + +/********************************************************************* + * sysfs attributes for batteries - + * definitions and helper functions + */ + +/* A custom device attribute struct which holds a battery number */ +struct bat_device_attribute { + struct device_attribute dev_attr; + int bat; +}; + +/** + * attr_get_bat - get the battery to which the attribute belongs + */ +static int attr_get_bat(struct device_attribute *attr) +{ + return container_of(attr, struct bat_device_attribute, dev_attr)->bat; +} + +/** + * show_tp_ec_bat_u16 - show an unsigned 16-bit battery attribute + * @arg0: specified 1st argument of EC raw to read + * @offset: byte offset in EC raw data + * @mul: correction factor to multiply by + * @na_msg: string to output is value not available (0xFFFFFFFF) + * @attr: battery attribute + * @buf: output buffer + * The 16-bit value is read from the EC, treated as unsigned, + * transformed as x->mul*x, and printed to the buffer. + * If the value is 0xFFFFFFFF and na_msg!=%NULL, na_msg is printed instead. + */ +static ssize_t show_tp_ec_bat_u16(u8 arg0, int offset, int mul, + const char *na_msg, + struct device_attribute *attr, char *buf) +{ + u16 val; + int ret = get_tp_ec_bat_16(arg0, offset, attr_get_bat(attr), &val); + if (ret) + return ret; + if (na_msg && val == 0xFFFF) + return sprintf(buf, "%s\n", na_msg); + else + return sprintf(buf, "%u\n", mul*(unsigned int)val); +} + +/** + * show_tp_ec_bat_s16 - show an signed 16-bit battery attribute + * @arg0: specified 1st argument of EC raw to read + * @offset: byte offset in EC raw data + * @mul: correction factor to multiply by + * @add: correction term to add after multiplication + * @attr: battery attribute + * @buf: output buffer + * The 16-bit value is read from the EC, treated as signed, + * transformed as x->mul*x+add, and printed to the buffer. + */ +static ssize_t show_tp_ec_bat_s16(u8 arg0, int offset, int mul, int add, + struct device_attribute *attr, char *buf) +{ + u16 val; + int ret = get_tp_ec_bat_16(arg0, offset, attr_get_bat(attr), &val); + if (ret) + return ret; + return sprintf(buf, "%d\n", mul*(s16)val+add); +} + +/** + * show_tp_ec_bat_str - show a string from EC battery status data + * @arg0: specified 1st argument of EC raw to read + * @offset: byte offset in EC raw data + * @maxlen: maximum string length + * @attr: battery attribute + * @buf: output buffer + */ +static ssize_t show_tp_ec_bat_str(u8 arg0, int offset, int maxlen, + struct device_attribute *attr, char *buf) +{ + int bat = attr_get_bat(attr); + u8 row[TP_CONTROLLER_ROW_LEN]; + int ret; + if (bat_has_status(bat) != 1) + return -ENXIO; + ret = read_tp_ec_row(arg0, bat, 0, row); + if (ret) + return ret; + strncpy(buf, (char *)row+offset, maxlen); + buf[maxlen] = 0; + strcat(buf, "\n"); + return strlen(buf); +} + +/** + * show_tp_ec_bat_power - show a power readout from EC battery status data + * @arg0: specified 1st argument of EC raw to read + * @offV: byte offset of voltage in EC raw data + * @offI: byte offset of current in EC raw data + * @attr: battery attribute + * @buf: output buffer + * Computes the power as current*voltage from the two given readout offsets. + */ +static ssize_t show_tp_ec_bat_power(u8 arg0, int offV, int offI, + struct device_attribute *attr, char *buf) +{ + u8 row[TP_CONTROLLER_ROW_LEN]; + int milliamp, millivolt, ret; + int bat = attr_get_bat(attr); + if (bat_has_status(bat) != 1) + return -ENXIO; + ret = read_tp_ec_row(1, bat, 0, row); + if (ret) + return ret; + millivolt = *(u16 *)(row+offV); + milliamp = *(s16 *)(row+offI); + return sprintf(buf, "%d\n", milliamp*millivolt/1000); /* units: mW */ +} + +/** + * show_tp_ec_bat_date - decode and show a date from EC battery status data + * @arg0: specified 1st argument of EC raw to read + * @offset: byte offset in EC raw data + * @attr: battery attribute + * @buf: output buffer + */ +static ssize_t show_tp_ec_bat_date(u8 arg0, int offset, + struct device_attribute *attr, char *buf) +{ + u8 row[TP_CONTROLLER_ROW_LEN]; + u16 v; + int ret; + int day, month, year; + int bat = attr_get_bat(attr); + if (bat_has_status(bat) != 1) + return -ENXIO; + ret = read_tp_ec_row(arg0, bat, 0, row); + if (ret) + return ret; + + /* Decode bit-packed: v = day | (month<<5) | ((year-1980)<<9) */ + v = *(u16 *)(row+offset); + day = v & 0x1F; + month = (v >> 5) & 0xF; + year = (v >> 9) + 1980; + + return sprintf(buf, "%04d-%02d-%02d\n", year, month, day); +} + + +/********************************************************************* + * sysfs attribute I/O for batteries - + * the actual attribute show/store functions + */ + +static ssize_t show_battery_start_charge_thresh(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int thresh; + int bat = attr_get_bat(attr); + int ret = get_thresh(bat, THRESH_START, &thresh); + if (ret) + return ret; + return sprintf(buf, "%d\n", thresh); /* units: percent */ +} + +static ssize_t show_battery_stop_charge_thresh(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int thresh; + int bat = attr_get_bat(attr); + int ret = get_thresh(bat, THRESH_STOP, &thresh); + if (ret) + return ret; + return sprintf(buf, "%d\n", thresh); /* units: percent */ +} + +/** + * store_battery_start_charge_thresh - store battery_start_charge_thresh attr + * Since this is a kernel<->user interface, we ensure a valid state for + * the hardware. We do this by clamping the requested threshold to the + * valid range and, if necessary, moving the other threshold so that + * it's MIN_THRESH_DELTA away from this one. + */ +static ssize_t store_battery_start_charge_thresh(struct device *dev, + struct device_attribute *attr, const char *buf, size_t count) +{ + int thresh, other_thresh, ret; + int bat = attr_get_bat(attr); + + if (sscanf(buf, "%d", &thresh) != 1 || thresh < 1 || thresh > 100) + return -EINVAL; + + if (thresh < MIN_THRESH_START) /* clamp up to MIN_THRESH_START */ + thresh = MIN_THRESH_START; + if (thresh > MAX_THRESH_START) /* clamp down to MAX_THRESH_START */ + thresh = MAX_THRESH_START; + + down(&smapi_mutex); + ret = get_thresh(bat, THRESH_STOP, &other_thresh); + if (ret != -EOPNOTSUPP && ret != -ENXIO) { + if (ret) /* other threshold is set? */ + goto out; + ret = get_real_thresh(bat, THRESH_START, NULL); + if (ret) /* this threshold is set? */ + goto out; + if (other_thresh < thresh+MIN_THRESH_DELTA) { + /* move other thresh to keep it above this one */ + ret = set_thresh(bat, THRESH_STOP, + thresh+MIN_THRESH_DELTA); + if (ret) + goto out; + } + } + ret = set_thresh(bat, THRESH_START, thresh); +out: + up(&smapi_mutex); + return count; + +} + +/** + * store_battery_stop_charge_thresh - store battery_stop_charge_thresh attr + * Since this is a kernel<->user interface, we ensure a valid state for + * the hardware. We do this by clamping the requested threshold to the + * valid range and, if necessary, moving the other threshold so that + * it's MIN_THRESH_DELTA away from this one. + */ +static ssize_t store_battery_stop_charge_thresh(struct device *dev, + struct device_attribute *attr, const char *buf, size_t count) +{ + int thresh, other_thresh, ret; + int bat = attr_get_bat(attr); + + if (sscanf(buf, "%d", &thresh) != 1 || thresh < 1 || thresh > 100) + return -EINVAL; + + if (thresh < MIN_THRESH_STOP) /* clamp up to MIN_THRESH_STOP */ + thresh = MIN_THRESH_STOP; + + down(&smapi_mutex); + ret = get_thresh(bat, THRESH_START, &other_thresh); + if (ret != -EOPNOTSUPP && ret != -ENXIO) { /* other threshold exists? */ + if (ret) + goto out; + /* this threshold exists? */ + ret = get_real_thresh(bat, THRESH_STOP, NULL); + if (ret) + goto out; + if (other_thresh >= thresh-MIN_THRESH_DELTA) { + /* move other thresh to be below this one */ + ret = set_thresh(bat, THRESH_START, + thresh-MIN_THRESH_DELTA); + if (ret) + goto out; + } + } + ret = set_thresh(bat, THRESH_STOP, thresh); +out: + up(&smapi_mutex); + return count; +} + +static ssize_t show_battery_inhibit_charge_minutes(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int minutes; + int bat = attr_get_bat(attr); + int ret = get_inhibit_charge_minutes(bat, &minutes); + if (ret) + return ret; + return sprintf(buf, "%d\n", minutes); /* units: minutes */ +} + +static ssize_t store_battery_inhibit_charge_minutes(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret; + int minutes; + int bat = attr_get_bat(attr); + if (sscanf(buf, "%d", &minutes) != 1 || minutes < 0) { + TPRINTK(KERN_ERR, "inhibit_charge_minutes: " + "must be a non-negative integer"); + return -EINVAL; + } + ret = set_inhibit_charge_minutes(bat, minutes); + if (ret) + return ret; + return count; +} + +static ssize_t show_battery_force_discharge(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int enabled; + int bat = attr_get_bat(attr); + int ret = get_force_discharge(bat, &enabled); + if (ret) + return ret; + return sprintf(buf, "%d\n", enabled); /* type: boolean */ +} + +static ssize_t store_battery_force_discharge(struct device *dev, + struct device_attribute *attr, const char *buf, size_t count) +{ + int ret; + int enabled; + int bat = attr_get_bat(attr); + if (sscanf(buf, "%d", &enabled) != 1 || enabled < 0 || enabled > 1) + return -EINVAL; + ret = set_force_discharge(bat, enabled); + if (ret) + return ret; + return count; +} + +static ssize_t show_battery_installed( + struct device *dev, struct device_attribute *attr, char *buf) +{ + int bat = attr_get_bat(attr); + int ret = power_device_present(bat); + if (ret < 0) + return ret; + return sprintf(buf, "%d\n", ret); /* type: boolean */ +} + +static ssize_t show_battery_state( + struct device *dev, struct device_attribute *attr, char *buf) +{ + u8 row[TP_CONTROLLER_ROW_LEN]; + const char *txt; + int ret; + int bat = attr_get_bat(attr); + if (bat_has_status(bat) != 1) + return sprintf(buf, "none\n"); + ret = read_tp_ec_row(1, bat, 0, row); + if (ret) + return ret; + switch (row[1] & 0xf0) { + case 0xc0: txt = "idle"; break; + case 0xd0: txt = "discharging"; break; + case 0xe0: txt = "charging"; break; + default: return sprintf(buf, "unknown (0x%x)\n", row[1]); + } + return sprintf(buf, "%s\n", txt); /* type: string from fixed set */ +} + +static ssize_t show_battery_manufacturer( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: string. SBS spec v1.1 p34: ManufacturerName() */ + return show_tp_ec_bat_str(4, 2, TP_CONTROLLER_ROW_LEN-2, attr, buf); +} + +static ssize_t show_battery_model( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: string. SBS spec v1.1 p34: DeviceName() */ + return show_tp_ec_bat_str(5, 2, TP_CONTROLLER_ROW_LEN-2, attr, buf); +} + +static ssize_t show_battery_barcoding( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: string */ + return show_tp_ec_bat_str(7, 2, TP_CONTROLLER_ROW_LEN-2, attr, buf); +} + +static ssize_t show_battery_chemistry( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: string. SBS spec v1.1 p34-35: DeviceChemistry() */ + return show_tp_ec_bat_str(6, 2, 5, attr, buf); +} + +static ssize_t show_battery_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV. SBS spec v1.1 p24: Voltage() */ + return show_tp_ec_bat_u16(1, 6, 1, NULL, attr, buf); +} + +static ssize_t show_battery_design_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV. SBS spec v1.1 p32: DesignVoltage() */ + return show_tp_ec_bat_u16(3, 4, 1, NULL, attr, buf); +} + +static ssize_t show_battery_charging_max_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV. SBS spec v1.1 p37,39: ChargingVoltage() */ + return show_tp_ec_bat_u16(9, 8, 1, NULL, attr, buf); +} + +static ssize_t show_battery_group0_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV */ + return show_tp_ec_bat_u16(0xA, 12, 1, NULL, attr, buf); +} + +static ssize_t show_battery_group1_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV */ + return show_tp_ec_bat_u16(0xA, 10, 1, NULL, attr, buf); +} + +static ssize_t show_battery_group2_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV */ + return show_tp_ec_bat_u16(0xA, 8, 1, NULL, attr, buf); +} + +static ssize_t show_battery_group3_voltage( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mV */ + return show_tp_ec_bat_u16(0xA, 6, 1, NULL, attr, buf); +} + +static ssize_t show_battery_current_now( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mA. SBS spec v1.1 p24: Current() */ + return show_tp_ec_bat_s16(1, 8, 1, 0, attr, buf); +} + +static ssize_t show_battery_current_avg( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mA. SBS spec v1.1 p24: AverageCurrent() */ + return show_tp_ec_bat_s16(1, 10, 1, 0, attr, buf); +} + +static ssize_t show_battery_charging_max_current( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mA. SBS spec v1.1 p36,38: ChargingCurrent() */ + return show_tp_ec_bat_s16(9, 6, 1, 0, attr, buf); +} + +static ssize_t show_battery_power_now( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mW. SBS spec v1.1: Voltage()*Current() */ + return show_tp_ec_bat_power(1, 6, 8, attr, buf); +} + +static ssize_t show_battery_power_avg( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mW. SBS spec v1.1: Voltage()*AverageCurrent() */ + return show_tp_ec_bat_power(1, 6, 10, attr, buf); +} + +static ssize_t show_battery_remaining_percent( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: percent. SBS spec v1.1 p25: RelativeStateOfCharge() */ + return show_tp_ec_bat_u16(1, 12, 1, NULL, attr, buf); +} + +static ssize_t show_battery_remaining_percent_error( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: percent. SBS spec v1.1 p25: MaxError() */ + return show_tp_ec_bat_u16(9, 4, 1, NULL, attr, buf); +} + +static ssize_t show_battery_remaining_charging_time( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: minutes. SBS spec v1.1 p27: AverageTimeToFull() */ + return show_tp_ec_bat_u16(2, 8, 1, "not_charging", attr, buf); +} + +static ssize_t show_battery_remaining_running_time( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: minutes. SBS spec v1.1 p27: RunTimeToEmpty() */ + return show_tp_ec_bat_u16(2, 6, 1, "not_discharging", attr, buf); +} + +static ssize_t show_battery_remaining_running_time_now( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: minutes. SBS spec v1.1 p27: RunTimeToEmpty() */ + return show_tp_ec_bat_u16(2, 4, 1, "not_discharging", attr, buf); +} + +static ssize_t show_battery_remaining_capacity( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mWh. SBS spec v1.1 p26. */ + return show_tp_ec_bat_u16(1, 14, 10, "", attr, buf); +} + +static ssize_t show_battery_last_full_capacity( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mWh. SBS spec v1.1 p26: FullChargeCapacity() */ + return show_tp_ec_bat_u16(2, 2, 10, "", attr, buf); +} + +static ssize_t show_battery_design_capacity( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: mWh. SBS spec v1.1 p32: DesignCapacity() */ + return show_tp_ec_bat_u16(3, 2, 10, "", attr, buf); +} + +static ssize_t show_battery_cycle_count( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: ordinal. SBS spec v1.1 p32: CycleCount() */ + return show_tp_ec_bat_u16(2, 12, 1, "", attr, buf); +} + +static ssize_t show_battery_temperature( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* units: millicelsius. SBS spec v1.1: Temperature()*10 */ + return show_tp_ec_bat_s16(1, 4, 100, -273100, attr, buf); +} + +static ssize_t show_battery_serial( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: int. SBS spec v1.1 p34: SerialNumber() */ + return show_tp_ec_bat_u16(3, 10, 1, "", attr, buf); +} + +static ssize_t show_battery_manufacture_date( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: YYYY-MM-DD. SBS spec v1.1 p34: ManufactureDate() */ + return show_tp_ec_bat_date(3, 8, attr, buf); +} + +static ssize_t show_battery_first_use_date( + struct device *dev, struct device_attribute *attr, char *buf) +{ + /* type: YYYY-MM-DD */ + return show_tp_ec_bat_date(8, 2, attr, buf); +} + +/** + * show_battery_dump - show the battery's dump attribute + * The dump attribute gives a hex dump of all EC readouts related to a + * battery. Some of the enumerated values don't really exist (i.e., the + * EC function just leaves them untouched); we use a kludge to detect and + * denote these. + */ +#define MIN_DUMP_ARG0 0x00 +#define MAX_DUMP_ARG0 0x0a /* 0x0b is useful too but hangs old EC firmware */ +static ssize_t show_battery_dump( + struct device *dev, struct device_attribute *attr, char *buf) +{ + int i; + char *p = buf; + int bat = attr_get_bat(attr); + u8 arg0; /* first argument to EC */ + u8 rowa[TP_CONTROLLER_ROW_LEN], + rowb[TP_CONTROLLER_ROW_LEN]; + const u8 junka = 0xAA, + junkb = 0x55; /* junk values for testing changes */ + int ret; + + for (arg0 = MIN_DUMP_ARG0; arg0 <= MAX_DUMP_ARG0; ++arg0) { + if ((p-buf) > PAGE_SIZE-TP_CONTROLLER_ROW_LEN*5) + return -ENOMEM; /* don't overflow sysfs buf */ + /* Read raw twice with different junk values, + * to detect unused output bytes which are left unchaged: */ + ret = read_tp_ec_row(arg0, bat, junka, rowa); + if (ret) + return ret; + ret = read_tp_ec_row(arg0, bat, junkb, rowb); + if (ret) + return ret; + for (i = 0; i < TP_CONTROLLER_ROW_LEN; i++) { + if (rowa[i] == junka && rowb[i] == junkb) + p += sprintf(p, "-- "); /* unused by EC */ + else + p += sprintf(p, "%02x ", rowa[i]); + } + p += sprintf(p, "\n"); + } + return p-buf; +} + + +/********************************************************************* + * sysfs attribute I/O, other than batteries + */ + +static ssize_t show_ac_connected( + struct device *dev, struct device_attribute *attr, char *buf) +{ + int ret = power_device_present(0xFF); + if (ret < 0) + return ret; + return sprintf(buf, "%d\n", ret); /* type: boolean */ +} + +/********************************************************************* + * The the "smapi_request" sysfs attribute executes a raw SMAPI call. + * You write to make a request and read to get the result. The state + * is saved globally rather than per fd (sysfs limitation), so + * simultaenous requests may get each other's results! So this is for + * development and debugging only. + */ +#define MAX_SMAPI_ATTR_ANSWER_LEN 128 +static char smapi_attr_answer[MAX_SMAPI_ATTR_ANSWER_LEN] = ""; + +static ssize_t show_smapi_request(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int ret = snprintf(buf, PAGE_SIZE, "%s", smapi_attr_answer); + smapi_attr_answer[0] = '\0'; + return ret; +} + +static ssize_t store_smapi_request(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + unsigned int inEBX, inECX, inEDI, inESI; + u32 outEBX, outECX, outEDX, outEDI, outESI; + const char *msg; + int ret; + if (sscanf(buf, "%x %x %x %x", &inEBX, &inECX, &inEDI, &inESI) != 4) { + smapi_attr_answer[0] = '\0'; + return -EINVAL; + } + ret = smapi_request( + inEBX, inECX, inEDI, inESI, + &outEBX, &outECX, &outEDX, &outEDI, &outESI, &msg); + snprintf(smapi_attr_answer, MAX_SMAPI_ATTR_ANSWER_LEN, + "%x %x %x %x %x %d '%s'\n", + (unsigned int)outEBX, (unsigned int)outECX, + (unsigned int)outEDX, (unsigned int)outEDI, + (unsigned int)outESI, ret, msg); + if (ret) + return ret; + else + return count; +} + +/********************************************************************* + * Power management: the embedded controller forgets the battery + * thresholds when the system is suspended to disk and unplugged from + * AC and battery, so we restore it upon resume. + */ + +static int saved_threshs[4] = {-1, -1, -1, -1}; /* -1 = don't know */ + +static int tp_suspend(struct platform_device *dev, pm_message_t state) +{ + int restore = (state.event == PM_EVENT_HIBERNATE || + state.event == PM_EVENT_FREEZE); + if (!restore || get_real_thresh(0, THRESH_STOP , &saved_threshs[0])) + saved_threshs[0] = -1; + if (!restore || get_real_thresh(0, THRESH_START, &saved_threshs[1])) + saved_threshs[1] = -1; + if (!restore || get_real_thresh(1, THRESH_STOP , &saved_threshs[2])) + saved_threshs[2] = -1; + if (!restore || get_real_thresh(1, THRESH_START, &saved_threshs[3])) + saved_threshs[3] = -1; + DPRINTK("suspend saved: %d %d %d %d", saved_threshs[0], + saved_threshs[1], saved_threshs[2], saved_threshs[3]); + return 0; +} + +static int tp_resume(struct platform_device *dev) +{ + DPRINTK("resume restoring: %d %d %d %d", saved_threshs[0], + saved_threshs[1], saved_threshs[2], saved_threshs[3]); + if (saved_threshs[0] >= 0) + set_real_thresh(0, THRESH_STOP , saved_threshs[0]); + if (saved_threshs[1] >= 0) + set_real_thresh(0, THRESH_START, saved_threshs[1]); + if (saved_threshs[2] >= 0) + set_real_thresh(1, THRESH_STOP , saved_threshs[2]); + if (saved_threshs[3] >= 0) + set_real_thresh(1, THRESH_START, saved_threshs[3]); + return 0; +} + + +/********************************************************************* + * Driver model + */ + +static struct platform_driver tp_driver = { + .suspend = tp_suspend, + .resume = tp_resume, + .driver = { + .name = "smapi", + .owner = THIS_MODULE + }, +}; + + +/********************************************************************* + * Sysfs device model + */ + +/* Attributes in /sys/devices/platform/smapi/ */ + +static DEVICE_ATTR(ac_connected, 0444, show_ac_connected, NULL); +static DEVICE_ATTR(smapi_request, 0600, show_smapi_request, + store_smapi_request); + +static struct attribute *tp_root_attributes[] = { + &dev_attr_ac_connected.attr, + &dev_attr_smapi_request.attr, + NULL +}; +static struct attribute_group tp_root_attribute_group = { + .attrs = tp_root_attributes +}; + +/* Attributes under /sys/devices/platform/smapi/BAT{0,1}/ : + * Every attribute needs to be defined (i.e., statically allocated) for + * each battery, and then referenced in the attribute list of each battery. + * We use preprocessor voodoo to avoid duplicating the list of attributes 4 + * times. The preprocessor output is just normal sysfs attributes code. + */ + +/** + * FOREACH_BAT_ATTR - invoke the given macros on all our battery attributes + * @_BAT: battery number (0 or 1) + * @_ATTR_RW: macro to invoke for each read/write attribute + * @_ATTR_R: macro to invoke for each read-only attribute + */ +#define FOREACH_BAT_ATTR(_BAT, _ATTR_RW, _ATTR_R) \ + _ATTR_RW(_BAT, start_charge_thresh) \ + _ATTR_RW(_BAT, stop_charge_thresh) \ + _ATTR_RW(_BAT, inhibit_charge_minutes) \ + _ATTR_RW(_BAT, force_discharge) \ + _ATTR_R(_BAT, installed) \ + _ATTR_R(_BAT, state) \ + _ATTR_R(_BAT, manufacturer) \ + _ATTR_R(_BAT, model) \ + _ATTR_R(_BAT, barcoding) \ + _ATTR_R(_BAT, chemistry) \ + _ATTR_R(_BAT, voltage) \ + _ATTR_R(_BAT, group0_voltage) \ + _ATTR_R(_BAT, group1_voltage) \ + _ATTR_R(_BAT, group2_voltage) \ + _ATTR_R(_BAT, group3_voltage) \ + _ATTR_R(_BAT, current_now) \ + _ATTR_R(_BAT, current_avg) \ + _ATTR_R(_BAT, charging_max_current) \ + _ATTR_R(_BAT, power_now) \ + _ATTR_R(_BAT, power_avg) \ + _ATTR_R(_BAT, remaining_percent) \ + _ATTR_R(_BAT, remaining_percent_error) \ + _ATTR_R(_BAT, remaining_charging_time) \ + _ATTR_R(_BAT, remaining_running_time) \ + _ATTR_R(_BAT, remaining_running_time_now) \ + _ATTR_R(_BAT, remaining_capacity) \ + _ATTR_R(_BAT, last_full_capacity) \ + _ATTR_R(_BAT, design_voltage) \ + _ATTR_R(_BAT, charging_max_voltage) \ + _ATTR_R(_BAT, design_capacity) \ + _ATTR_R(_BAT, cycle_count) \ + _ATTR_R(_BAT, temperature) \ + _ATTR_R(_BAT, serial) \ + _ATTR_R(_BAT, manufacture_date) \ + _ATTR_R(_BAT, first_use_date) \ + _ATTR_R(_BAT, dump) + +/* Define several macros we will feed into FOREACH_BAT_ATTR: */ + +#define DEFINE_BAT_ATTR_RW(_BAT,_NAME) \ + static struct bat_device_attribute dev_attr_##_NAME##_##_BAT = { \ + .dev_attr = __ATTR(_NAME, 0644, show_battery_##_NAME, \ + store_battery_##_NAME), \ + .bat = _BAT \ + }; + +#define DEFINE_BAT_ATTR_R(_BAT,_NAME) \ + static struct bat_device_attribute dev_attr_##_NAME##_##_BAT = { \ + .dev_attr = __ATTR(_NAME, 0644, show_battery_##_NAME, 0), \ + .bat = _BAT \ + }; + +#define REF_BAT_ATTR(_BAT,_NAME) \ + &dev_attr_##_NAME##_##_BAT.dev_attr.attr, + +/* This provide all attributes for one battery: */ + +#define PROVIDE_BAT_ATTRS(_BAT) \ + FOREACH_BAT_ATTR(_BAT, DEFINE_BAT_ATTR_RW, DEFINE_BAT_ATTR_R) \ + static struct attribute *tp_bat##_BAT##_attributes[] = { \ + FOREACH_BAT_ATTR(_BAT, REF_BAT_ATTR, REF_BAT_ATTR) \ + NULL \ + }; \ + static struct attribute_group tp_bat##_BAT##_attribute_group = { \ + .name = "BAT" #_BAT, \ + .attrs = tp_bat##_BAT##_attributes \ + }; + +/* Finally genereate the attributes: */ + +PROVIDE_BAT_ATTRS(0) +PROVIDE_BAT_ATTRS(1) + +/* List of attribute groups */ + +static struct attribute_group *attr_groups[] = { + &tp_root_attribute_group, + &tp_bat0_attribute_group, + &tp_bat1_attribute_group, + NULL +}; + + +/********************************************************************* + * Init and cleanup + */ + +static struct attribute_group **next_attr_group; /* next to register */ + +static int __init tp_init(void) +{ + int ret; + printk(KERN_INFO "tp_smapi " TP_VERSION " loading...\n"); + + ret = find_smapi_port(); + if (ret < 0) + goto err; + else + smapi_port = ret; + + if (!request_region(smapi_port, 1, "smapi")) { + printk(KERN_ERR "tp_smapi cannot claim port 0x%x\n", + smapi_port); + ret = -ENXIO; + goto err; + } + + if (!request_region(SMAPI_PORT2, 1, "smapi")) { + printk(KERN_ERR "tp_smapi cannot claim port 0x%x\n", + SMAPI_PORT2); + ret = -ENXIO; + goto err_port1; + } + + ret = platform_driver_register(&tp_driver); + if (ret) + goto err_port2; + + pdev = platform_device_alloc("smapi", -1); + if (!pdev) { + ret = -ENOMEM; + goto err_driver; + } + + ret = platform_device_add(pdev); + if (ret) + goto err_device_free; + + for (next_attr_group = attr_groups; *next_attr_group; + ++next_attr_group) { + ret = sysfs_create_group(&pdev->dev.kobj, *next_attr_group); + if (ret) + goto err_attr; + } + + printk(KERN_INFO "tp_smapi successfully loaded (smapi_port=0x%x).\n", + smapi_port); + return 0; + +err_attr: + while (--next_attr_group >= attr_groups) + sysfs_remove_group(&pdev->dev.kobj, *next_attr_group); + platform_device_unregister(pdev); +err_device_free: + platform_device_put(pdev); +err_driver: + platform_driver_unregister(&tp_driver); +err_port2: + release_region(SMAPI_PORT2, 1); +err_port1: + release_region(smapi_port, 1); +err: + printk(KERN_ERR "tp_smapi init failed (ret=%d)!\n", ret); + return ret; +} + +static void __exit tp_exit(void) +{ + while (next_attr_group && --next_attr_group >= attr_groups) + sysfs_remove_group(&pdev->dev.kobj, *next_attr_group); + platform_device_unregister(pdev); + platform_driver_unregister(&tp_driver); + release_region(SMAPI_PORT2, 1); + if (smapi_port) + release_region(smapi_port, 1); + + printk(KERN_INFO "tp_smapi unloaded.\n"); +} + +module_init(tp_init); +module_exit(tp_exit); diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig index d145e0d..4a62250 100644 --- a/drivers/scsi/Kconfig +++ b/drivers/scsi/Kconfig @@ -1613,4 +1613,6 @@ source "drivers/scsi/device_handler/Kconfig" source "drivers/scsi/osd/Kconfig" +source "drivers/scsi/vhba/Kconfig" + endmenu diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile index 93dbe58..0d83856 100644 --- a/drivers/scsi/Makefile +++ b/drivers/scsi/Makefile @@ -154,6 +154,7 @@ obj-$(CONFIG_SCSI_ENCLOSURE) += ses.o obj-$(CONFIG_SCSI_OSD_INITIATOR) += osd/ obj-$(CONFIG_SCSI_HISI_SAS) += hisi_sas/ +obj-$(CONFIG_VHBA) += vhba/ # This goes last, so that "real" scsi devices probe earlier obj-$(CONFIG_SCSI_DEBUG) += scsi_debug.o diff --git b/drivers/scsi/vhba/Kconfig b/drivers/scsi/vhba/Kconfig new file mode 100644 index 0000000..7ccb7d8 --- /dev/null +++ b/drivers/scsi/vhba/Kconfig @@ -0,0 +1,9 @@ +config VHBA + tristate "Virtual (SCSI) Host Bus Adapter" + depends on SCSI + ---help--- + This is the in-kernel part of CDEmu, a CD/DVD-ROM device + emulator. + + This driver can also be built as a module. If so, the module + will be called vhba. diff --git b/drivers/scsi/vhba/Makefile b/drivers/scsi/vhba/Makefile new file mode 100644 index 0000000..a2a3f9d --- /dev/null +++ b/drivers/scsi/vhba/Makefile @@ -0,0 +1,4 @@ +VHBA_VERSION := 20170610 + +obj-$(CONFIG_VHBA) += vhba.o +ccflags-y := -DVHBA_VERSION=\"$(VHBA_VERSION)\" -Werror diff --git b/drivers/scsi/vhba/vhba.c b/drivers/scsi/vhba/vhba.c new file mode 100644 index 0000000..ff30e4c --- /dev/null +++ b/drivers/scsi/vhba/vhba.c @@ -0,0 +1,1076 @@ +/* + * vhba.c + * + * Copyright (C) 2007-2012 Chia-I Wu + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ + +#include + +#include +#include +#include +#include +#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 11, 0) +#include +#else +#include +#endif +#include +#include +#include +#include +#ifdef CONFIG_COMPAT +#include +#endif +#include +#include +#include +#include +#include + +/* scatterlist.page_link and sg_page() were introduced in 2.6.24 */ +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 24) +#define USE_SG_PAGE +#include +#endif + +MODULE_AUTHOR("Chia-I Wu"); +MODULE_VERSION(VHBA_VERSION); +MODULE_DESCRIPTION("Virtual SCSI HBA"); +MODULE_LICENSE("GPL"); + +#ifdef DEBUG +#define DPRINTK(fmt, args...) printk(KERN_DEBUG "%s: " fmt, __FUNCTION__, ## args) +#else +#define DPRINTK(fmt, args...) +#endif + +/* scmd_dbg was introduced in 3.15 */ +#ifndef scmd_dbg +#define scmd_dbg(scmd, fmt, a...) \ + dev_dbg(&(scmd)->device->sdev_gendev, fmt, ##a) +#endif + +#ifndef scmd_warn +#define scmd_warn(scmd, fmt, a...) \ + dev_warn(&(scmd)->device->sdev_gendev, fmt, ##a) +#endif + +#define VHBA_MAX_SECTORS_PER_IO 256 +#define VHBA_MAX_ID 32 +#define VHBA_CAN_QUEUE 32 +#define VHBA_INVALID_ID VHBA_MAX_ID + +#define DATA_TO_DEVICE(dir) ((dir) == DMA_TO_DEVICE || (dir) == DMA_BIDIRECTIONAL) +#define DATA_FROM_DEVICE(dir) ((dir) == DMA_FROM_DEVICE || (dir) == DMA_BIDIRECTIONAL) + + +/* SCSI macros were introduced in 2.6.23 */ +#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 23) +#define scsi_sg_count(cmd) ((cmd)->use_sg) +#define scsi_sglist(cmd) ((cmd)->request_buffer) +#define scsi_bufflen(cmd) ((cmd)->request_bufflen) +#define scsi_set_resid(cmd, to_read) {(cmd)->resid = (to_read);} +#endif + +/* 1-argument form of k[un]map_atomic was introduced in 2.6.37-rc1; + 2-argument form was deprecated in 3.4-rc1 */ +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37) +#define vhba_kmap_atomic kmap_atomic +#define vhba_kunmap_atomic kunmap_atomic +#else +#define vhba_kmap_atomic(page) kmap_atomic(page, KM_USER0) +#define vhba_kunmap_atomic(page) kunmap_atomic(page, KM_USER0) +#endif + + +enum vhba_req_state { + VHBA_REQ_FREE, + VHBA_REQ_PENDING, + VHBA_REQ_READING, + VHBA_REQ_SENT, + VHBA_REQ_WRITING, +}; + +struct vhba_command { + struct scsi_cmnd *cmd; + int status; + struct list_head entry; +}; + +struct vhba_device { + uint id; + spinlock_t cmd_lock; + struct list_head cmd_list; + wait_queue_head_t cmd_wq; + atomic_t refcnt; +}; + +struct vhba_host { + struct Scsi_Host *shost; + spinlock_t cmd_lock; + int cmd_next; + struct vhba_command commands[VHBA_CAN_QUEUE]; + spinlock_t dev_lock; + struct vhba_device *devices[VHBA_MAX_ID]; + int num_devices; + DECLARE_BITMAP(chgmap, VHBA_MAX_ID); + int chgtype[VHBA_MAX_ID]; + struct work_struct scan_devices; +}; + +#define MAX_COMMAND_SIZE 16 + +struct vhba_request { + __u32 tag; + __u32 lun; + __u8 cdb[MAX_COMMAND_SIZE]; + __u8 cdb_len; + __u32 data_len; +}; + +struct vhba_response { + __u32 tag; + __u32 status; + __u32 data_len; +}; + +static struct vhba_command *vhba_alloc_command (void); +static void vhba_free_command (struct vhba_command *vcmd); + +static struct platform_device vhba_platform_device; + +static struct vhba_device *vhba_device_alloc (void) +{ + struct vhba_device *vdev; + + vdev = kzalloc(sizeof(struct vhba_device), GFP_KERNEL); + if (!vdev) { + return NULL; + } + + vdev->id = VHBA_INVALID_ID; + spin_lock_init(&vdev->cmd_lock); + INIT_LIST_HEAD(&vdev->cmd_list); + init_waitqueue_head(&vdev->cmd_wq); + atomic_set(&vdev->refcnt, 1); + + return vdev; +} + +static void vhba_device_put (struct vhba_device *vdev) +{ + if (atomic_dec_and_test(&vdev->refcnt)) { + kfree(vdev); + } +} + +static struct vhba_device *vhba_device_get (struct vhba_device *vdev) +{ + atomic_inc(&vdev->refcnt); + + return vdev; +} + +static int vhba_device_queue (struct vhba_device *vdev, struct scsi_cmnd *cmd) +{ + struct vhba_command *vcmd; + unsigned long flags; + + vcmd = vhba_alloc_command(); + if (!vcmd) { + return SCSI_MLQUEUE_HOST_BUSY; + } + + vcmd->cmd = cmd; + + spin_lock_irqsave(&vdev->cmd_lock, flags); + list_add_tail(&vcmd->entry, &vdev->cmd_list); + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + wake_up_interruptible(&vdev->cmd_wq); + + return 0; +} + +static int vhba_device_dequeue (struct vhba_device *vdev, struct scsi_cmnd *cmd) +{ + struct vhba_command *vcmd; + int retval; + unsigned long flags; + + spin_lock_irqsave(&vdev->cmd_lock, flags); + list_for_each_entry(vcmd, &vdev->cmd_list, entry) { + if (vcmd->cmd == cmd) { + list_del_init(&vcmd->entry); + break; + } + } + + /* command not found */ + if (&vcmd->entry == &vdev->cmd_list) { + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + return SUCCESS; + } + + while (vcmd->status == VHBA_REQ_READING || vcmd->status == VHBA_REQ_WRITING) { + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + scmd_dbg(cmd, "wait for I/O before aborting\n"); + schedule_timeout(1); + spin_lock_irqsave(&vdev->cmd_lock, flags); + } + + retval = (vcmd->status == VHBA_REQ_SENT) ? FAILED : SUCCESS; + + vhba_free_command(vcmd); + + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + return retval; +} + +static inline void vhba_scan_devices_add (struct vhba_host *vhost, int id) +{ + struct scsi_device *sdev; + + sdev = scsi_device_lookup(vhost->shost, 0, id, 0); + if (!sdev) { + scsi_add_device(vhost->shost, 0, id, 0); + } else { + dev_warn(&vhost->shost->shost_gendev, "tried to add an already-existing device 0:%d:0!\n", id); + scsi_device_put(sdev); + } +} + +static inline void vhba_scan_devices_remove (struct vhba_host *vhost, int id) +{ + struct scsi_device *sdev; + + sdev = scsi_device_lookup(vhost->shost, 0, id, 0); + if (sdev) { + scsi_remove_device(sdev); + scsi_device_put(sdev); + } else { + dev_warn(&vhost->shost->shost_gendev, "tried to remove non-existing device 0:%d:0!\n", id); + } +} + +static void vhba_scan_devices (struct work_struct *work) +{ + struct vhba_host *vhost = container_of(work, struct vhba_host, scan_devices); + unsigned long flags; + int id, change, exists; + + while (1) { + spin_lock_irqsave(&vhost->dev_lock, flags); + + id = find_first_bit(vhost->chgmap, vhost->shost->max_id); + if (id >= vhost->shost->max_id) { + spin_unlock_irqrestore(&vhost->dev_lock, flags); + break; + } + change = vhost->chgtype[id]; + exists = vhost->devices[id] != NULL; + + vhost->chgtype[id] = 0; + clear_bit(id, vhost->chgmap); + + spin_unlock_irqrestore(&vhost->dev_lock, flags); + + if (change < 0) { + dev_dbg(&vhost->shost->shost_gendev, "trying to remove target 0:%d:0\n", id); + vhba_scan_devices_remove(vhost, id); + } else if (change > 0) { + dev_dbg(&vhost->shost->shost_gendev, "trying to add target 0:%d:0\n", id); + vhba_scan_devices_add(vhost, id); + } else { + /* quick sequence of add/remove or remove/add; we determine + which one it was by checking if device structure exists */ + if (exists) { + /* remove followed by add: remove and (re)add */ + dev_dbg(&vhost->shost->shost_gendev, "trying to (re)add target 0:%d:0\n", id); + vhba_scan_devices_remove(vhost, id); + vhba_scan_devices_add(vhost, id); + } else { + /* add followed by remove: no-op */ + dev_dbg(&vhost->shost->shost_gendev, "no-op for target 0:%d:0\n", id); + } + } + } +} + +static int vhba_add_device (struct vhba_device *vdev) +{ + struct vhba_host *vhost; + int i; + unsigned long flags; + + vhost = platform_get_drvdata(&vhba_platform_device); + + vhba_device_get(vdev); + + spin_lock_irqsave(&vhost->dev_lock, flags); + if (vhost->num_devices >= vhost->shost->max_id) { + spin_unlock_irqrestore(&vhost->dev_lock, flags); + vhba_device_put(vdev); + return -EBUSY; + } + + for (i = 0; i < vhost->shost->max_id; i++) { + if (vhost->devices[i] == NULL) { + vdev->id = i; + vhost->devices[i] = vdev; + vhost->num_devices++; + set_bit(vdev->id, vhost->chgmap); + vhost->chgtype[vdev->id]++; + break; + } + } + spin_unlock_irqrestore(&vhost->dev_lock, flags); + + schedule_work(&vhost->scan_devices); + + return 0; +} + +static int vhba_remove_device (struct vhba_device *vdev) +{ + struct vhba_host *vhost; + unsigned long flags; + + vhost = platform_get_drvdata(&vhba_platform_device); + + spin_lock_irqsave(&vhost->dev_lock, flags); + set_bit(vdev->id, vhost->chgmap); + vhost->chgtype[vdev->id]--; + vhost->devices[vdev->id] = NULL; + vhost->num_devices--; + vdev->id = VHBA_INVALID_ID; + spin_unlock_irqrestore(&vhost->dev_lock, flags); + + vhba_device_put(vdev); + + schedule_work(&vhost->scan_devices); + + return 0; +} + +static struct vhba_device *vhba_lookup_device (int id) +{ + struct vhba_host *vhost; + struct vhba_device *vdev = NULL; + unsigned long flags; + + vhost = platform_get_drvdata(&vhba_platform_device); + + if (likely(id < vhost->shost->max_id)) { + spin_lock_irqsave(&vhost->dev_lock, flags); + vdev = vhost->devices[id]; + if (vdev) { + vdev = vhba_device_get(vdev); + } + + spin_unlock_irqrestore(&vhost->dev_lock, flags); + } + + return vdev; +} + +static struct vhba_command *vhba_alloc_command (void) +{ + struct vhba_host *vhost; + struct vhba_command *vcmd; + unsigned long flags; + int i; + + vhost = platform_get_drvdata(&vhba_platform_device); + + spin_lock_irqsave(&vhost->cmd_lock, flags); + + vcmd = vhost->commands + vhost->cmd_next++; + if (vcmd->status != VHBA_REQ_FREE) { + for (i = 0; i < vhost->shost->can_queue; i++) { + vcmd = vhost->commands + i; + + if (vcmd->status == VHBA_REQ_FREE) { + vhost->cmd_next = i + 1; + break; + } + } + + if (i == vhost->shost->can_queue) { + vcmd = NULL; + } + } + + if (vcmd) { + vcmd->status = VHBA_REQ_PENDING; + } + + vhost->cmd_next %= vhost->shost->can_queue; + + spin_unlock_irqrestore(&vhost->cmd_lock, flags); + + return vcmd; +} + +static void vhba_free_command (struct vhba_command *vcmd) +{ + struct vhba_host *vhost; + unsigned long flags; + + vhost = platform_get_drvdata(&vhba_platform_device); + + spin_lock_irqsave(&vhost->cmd_lock, flags); + vcmd->status = VHBA_REQ_FREE; + spin_unlock_irqrestore(&vhost->cmd_lock, flags); +} + +static int vhba_queuecommand_lck (struct scsi_cmnd *cmd, void (*done)(struct scsi_cmnd *)) +{ + struct vhba_device *vdev; + int retval; + + scmd_dbg(cmd, "queue %lu\n", cmd->serial_number); + + vdev = vhba_lookup_device(cmd->device->id); + if (!vdev) { + scmd_dbg(cmd, "no such device\n"); + + cmd->result = DID_NO_CONNECT << 16; + done(cmd); + + return 0; + } + + cmd->scsi_done = done; + retval = vhba_device_queue(vdev, cmd); + + vhba_device_put(vdev); + + return retval; +} + +#ifdef DEF_SCSI_QCMD +DEF_SCSI_QCMD(vhba_queuecommand) +#else +#define vhba_queuecommand vhba_queuecommand_lck +#endif + +static int vhba_abort (struct scsi_cmnd *cmd) +{ + struct vhba_device *vdev; + int retval = SUCCESS; + + scmd_warn(cmd, "abort %lu\n", cmd->serial_number); + + vdev = vhba_lookup_device(cmd->device->id); + if (vdev) { + retval = vhba_device_dequeue(vdev, cmd); + vhba_device_put(vdev); + } else { + cmd->result = DID_NO_CONNECT << 16; + } + + return retval; +} + +static struct scsi_host_template vhba_template = { + .module = THIS_MODULE, + .name = "vhba", + .proc_name = "vhba", + .queuecommand = vhba_queuecommand, + .eh_abort_handler = vhba_abort, + .can_queue = VHBA_CAN_QUEUE, + .this_id = -1, + .cmd_per_lun = 1, + .max_sectors = VHBA_MAX_SECTORS_PER_IO, + .sg_tablesize = 256, +}; + +static ssize_t do_request (struct scsi_cmnd *cmd, char __user *buf, size_t buf_len) +{ + struct vhba_request vreq; + ssize_t ret; + + scmd_dbg(cmd, "request %lu, cdb 0x%x, bufflen %d, use_sg %d\n", + cmd->serial_number, cmd->cmnd[0], scsi_bufflen(cmd), scsi_sg_count(cmd)); + + ret = sizeof(vreq); + if (DATA_TO_DEVICE(cmd->sc_data_direction)) { + ret += scsi_bufflen(cmd); + } + + if (ret > buf_len) { + scmd_warn(cmd, "buffer too small (%zd < %zd) for a request\n", buf_len, ret); + return -EIO; + } + + vreq.tag = cmd->serial_number; + vreq.lun = cmd->device->lun; + memcpy(vreq.cdb, cmd->cmnd, MAX_COMMAND_SIZE); + vreq.cdb_len = cmd->cmd_len; + vreq.data_len = scsi_bufflen(cmd); + + if (copy_to_user(buf, &vreq, sizeof(vreq))) { + return -EFAULT; + } + + if (DATA_TO_DEVICE(cmd->sc_data_direction) && vreq.data_len) { + buf += sizeof(vreq); + + if (scsi_sg_count(cmd)) { + unsigned char buf_stack[64]; + unsigned char *kaddr, *uaddr, *kbuf; + struct scatterlist *sg = scsi_sglist(cmd); + int i; + + uaddr = (unsigned char *) buf; + + if (vreq.data_len > 64) { + kbuf = kmalloc(PAGE_SIZE, GFP_KERNEL); + } else { + kbuf = buf_stack; + } + + for (i = 0; i < scsi_sg_count(cmd); i++) { + size_t len = sg[i].length; + +#ifdef USE_SG_PAGE + kaddr = vhba_kmap_atomic(sg_page(&sg[i])); +#else + kaddr = vhba_kmap_atomic(sg[i].page); +#endif + memcpy(kbuf, kaddr + sg[i].offset, len); + vhba_kunmap_atomic(kaddr); + + if (copy_to_user(uaddr, kbuf, len)) { + if (kbuf != buf_stack) { + kfree(kbuf); + } + return -EFAULT; + } + uaddr += len; + } + + if (kbuf != buf_stack) { + kfree(kbuf); + } + } else { + if (copy_to_user(buf, scsi_sglist(cmd), vreq.data_len)) { + return -EFAULT; + } + } + } + + return ret; +} + +static ssize_t do_response (struct scsi_cmnd *cmd, const char __user *buf, size_t buf_len, struct vhba_response *res) +{ + ssize_t ret = 0; + + scmd_dbg(cmd, "response %lu, status %x, data len %d, use_sg %d\n", + cmd->serial_number, res->status, res->data_len, scsi_sg_count(cmd)); + + if (res->status) { + unsigned char sense_stack[SCSI_SENSE_BUFFERSIZE]; + + if (res->data_len > SCSI_SENSE_BUFFERSIZE) { + scmd_warn(cmd, "truncate sense (%d < %d)", SCSI_SENSE_BUFFERSIZE, res->data_len); + res->data_len = SCSI_SENSE_BUFFERSIZE; + } + + /* Copy via temporary buffer on stack in order to avoid problems + with PAX on grsecurity-enabled kernels */ + if (copy_from_user(sense_stack, buf, res->data_len)) { + return -EFAULT; + } + memcpy(cmd->sense_buffer, sense_stack, res->data_len); + + cmd->result = res->status; + + ret += res->data_len; + } else if (DATA_FROM_DEVICE(cmd->sc_data_direction) && scsi_bufflen(cmd)) { + size_t to_read; + + if (res->data_len > scsi_bufflen(cmd)) { + scmd_warn(cmd, "truncate data (%d < %d)\n", scsi_bufflen(cmd), res->data_len); + res->data_len = scsi_bufflen(cmd); + } + + to_read = res->data_len; + + if (scsi_sg_count(cmd)) { + unsigned char buf_stack[64]; + unsigned char *kaddr, *uaddr, *kbuf; + struct scatterlist *sg = scsi_sglist(cmd); + int i; + + uaddr = (unsigned char *)buf; + + if (res->data_len > 64) { + kbuf = kmalloc(PAGE_SIZE, GFP_KERNEL); + } else { + kbuf = buf_stack; + } + + for (i = 0; i < scsi_sg_count(cmd); i++) { + size_t len = (sg[i].length < to_read) ? sg[i].length : to_read; + + if (copy_from_user(kbuf, uaddr, len)) { + if (kbuf != buf_stack) { + kfree(kbuf); + } + return -EFAULT; + } + uaddr += len; + +#ifdef USE_SG_PAGE + kaddr = vhba_kmap_atomic(sg_page(&sg[i])); +#else + kaddr = vhba_kmap_atomic(sg[i].page); +#endif + memcpy(kaddr + sg[i].offset, kbuf, len); + vhba_kunmap_atomic(kaddr); + + to_read -= len; + if (to_read == 0) { + break; + } + } + + if (kbuf != buf_stack) { + kfree(kbuf); + } + } else { + if (copy_from_user(scsi_sglist(cmd), buf, res->data_len)) { + return -EFAULT; + } + + to_read -= res->data_len; + } + + scsi_set_resid(cmd, to_read); + + ret += res->data_len - to_read; + } + + return ret; +} + +static inline struct vhba_command *next_command (struct vhba_device *vdev) +{ + struct vhba_command *vcmd; + + list_for_each_entry(vcmd, &vdev->cmd_list, entry) { + if (vcmd->status == VHBA_REQ_PENDING) { + break; + } + } + + if (&vcmd->entry == &vdev->cmd_list) { + vcmd = NULL; + } + + return vcmd; +} + +static inline struct vhba_command *match_command (struct vhba_device *vdev, u32 tag) +{ + struct vhba_command *vcmd; + + list_for_each_entry(vcmd, &vdev->cmd_list, entry) { + if (vcmd->cmd->serial_number == tag) { + break; + } + } + + if (&vcmd->entry == &vdev->cmd_list) { + vcmd = NULL; + } + + return vcmd; +} + +static struct vhba_command *wait_command (struct vhba_device *vdev, unsigned long flags) +{ + struct vhba_command *vcmd; + DEFINE_WAIT(wait); + + while (!(vcmd = next_command(vdev))) { + if (signal_pending(current)) { + break; + } + + prepare_to_wait(&vdev->cmd_wq, &wait, TASK_INTERRUPTIBLE); + + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + schedule(); + + spin_lock_irqsave(&vdev->cmd_lock, flags); + } + + finish_wait(&vdev->cmd_wq, &wait); + if (vcmd) { + vcmd->status = VHBA_REQ_READING; + } + + return vcmd; +} + +static ssize_t vhba_ctl_read (struct file *file, char __user *buf, size_t buf_len, loff_t *offset) +{ + struct vhba_device *vdev; + struct vhba_command *vcmd; + ssize_t ret; + unsigned long flags; + + vdev = file->private_data; + + /* Get next command */ + if (file->f_flags & O_NONBLOCK) { + /* Non-blocking variant */ + spin_lock_irqsave(&vdev->cmd_lock, flags); + vcmd = next_command(vdev); + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + if (!vcmd) { + return -EWOULDBLOCK; + } + } else { + /* Blocking variant */ + spin_lock_irqsave(&vdev->cmd_lock, flags); + vcmd = wait_command(vdev, flags); + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + if (!vcmd) { + return -ERESTARTSYS; + } + } + + ret = do_request(vcmd->cmd, buf, buf_len); + + spin_lock_irqsave(&vdev->cmd_lock, flags); + if (ret >= 0) { + vcmd->status = VHBA_REQ_SENT; + *offset += ret; + } else { + vcmd->status = VHBA_REQ_PENDING; + } + + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + return ret; +} + +static ssize_t vhba_ctl_write (struct file *file, const char __user *buf, size_t buf_len, loff_t *offset) +{ + struct vhba_device *vdev; + struct vhba_command *vcmd; + struct vhba_response res; + ssize_t ret; + unsigned long flags; + + if (buf_len < sizeof(res)) { + return -EIO; + } + + if (copy_from_user(&res, buf, sizeof(res))) { + return -EFAULT; + } + + vdev = file->private_data; + + spin_lock_irqsave(&vdev->cmd_lock, flags); + vcmd = match_command(vdev, res.tag); + if (!vcmd || vcmd->status != VHBA_REQ_SENT) { + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + DPRINTK("not expecting response\n"); + return -EIO; + } + vcmd->status = VHBA_REQ_WRITING; + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + ret = do_response(vcmd->cmd, buf + sizeof(res), buf_len - sizeof(res), &res); + + spin_lock_irqsave(&vdev->cmd_lock, flags); + if (ret >= 0) { + vcmd->cmd->scsi_done(vcmd->cmd); + ret += sizeof(res); + + /* don't compete with vhba_device_dequeue */ + if (!list_empty(&vcmd->entry)) { + list_del_init(&vcmd->entry); + vhba_free_command(vcmd); + } + } else { + vcmd->status = VHBA_REQ_SENT; + } + + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + return ret; +} + +static long vhba_ctl_ioctl (struct file *file, unsigned int cmd, unsigned long arg) +{ + struct vhba_device *vdev = file->private_data; + struct vhba_host *vhost; + struct scsi_device *sdev; + + switch (cmd) { + case 0xBEEF001: { + vhost = platform_get_drvdata(&vhba_platform_device); + sdev = scsi_device_lookup(vhost->shost, 0, vdev->id, 0); + + if (sdev) { + int id[4] = { + sdev->host->host_no, + sdev->channel, + sdev->id, + sdev->lun + }; + + scsi_device_put(sdev); + + if (copy_to_user((void *)arg, id, sizeof(id))) { + return -EFAULT; + } + + return 0; + } else { + return -ENODEV; + } + } + } + + return -ENOTTY; +} + +#ifdef CONFIG_COMPAT +static long vhba_ctl_compat_ioctl (struct file *file, unsigned int cmd, unsigned long arg) +{ + unsigned long compat_arg = (unsigned long)compat_ptr(arg); + return vhba_ctl_ioctl(file, cmd, compat_arg); +} +#endif + +static unsigned int vhba_ctl_poll (struct file *file, poll_table *wait) +{ + struct vhba_device *vdev = file->private_data; + unsigned int mask = 0; + unsigned long flags; + + poll_wait(file, &vdev->cmd_wq, wait); + + spin_lock_irqsave(&vdev->cmd_lock, flags); + if (next_command(vdev)) { + mask |= POLLIN | POLLRDNORM; + } + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + return mask; +} + +static int vhba_ctl_open (struct inode *inode, struct file *file) +{ + struct vhba_device *vdev; + int retval; + + DPRINTK("open\n"); + + /* check if vhba is probed */ + if (!platform_get_drvdata(&vhba_platform_device)) { + return -ENODEV; + } + + vdev = vhba_device_alloc(); + if (!vdev) { + return -ENOMEM; + } + + if (!(retval = vhba_add_device(vdev))) { + file->private_data = vdev; + } + + vhba_device_put(vdev); + + return retval; +} + +static int vhba_ctl_release (struct inode *inode, struct file *file) +{ + struct vhba_device *vdev; + struct vhba_command *vcmd; + unsigned long flags; + + DPRINTK("release\n"); + + vdev = file->private_data; + + vhba_device_get(vdev); + vhba_remove_device(vdev); + + spin_lock_irqsave(&vdev->cmd_lock, flags); + list_for_each_entry(vcmd, &vdev->cmd_list, entry) { + WARN_ON(vcmd->status == VHBA_REQ_READING || vcmd->status == VHBA_REQ_WRITING); + + scmd_warn(vcmd->cmd, "device released with command %lu\n", vcmd->cmd->serial_number); + vcmd->cmd->result = DID_NO_CONNECT << 16; + vcmd->cmd->scsi_done(vcmd->cmd); + + vhba_free_command(vcmd); + } + INIT_LIST_HEAD(&vdev->cmd_list); + spin_unlock_irqrestore(&vdev->cmd_lock, flags); + + vhba_device_put(vdev); + + return 0; +} + +static struct file_operations vhba_ctl_fops = { + .owner = THIS_MODULE, + .open = vhba_ctl_open, + .release = vhba_ctl_release, + .read = vhba_ctl_read, + .write = vhba_ctl_write, + .poll = vhba_ctl_poll, + .unlocked_ioctl = vhba_ctl_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = vhba_ctl_compat_ioctl, +#endif +}; + +static struct miscdevice vhba_miscdev = { + .minor = MISC_DYNAMIC_MINOR, + .name = "vhba_ctl", + .fops = &vhba_ctl_fops, +}; + +static int vhba_probe (struct platform_device *pdev) +{ + struct Scsi_Host *shost; + struct vhba_host *vhost; + int i; + + shost = scsi_host_alloc(&vhba_template, sizeof(struct vhba_host)); + if (!shost) { + return -ENOMEM; + } + + shost->max_id = VHBA_MAX_ID; + /* we don't support lun > 0 */ + shost->max_lun = 1; + shost->max_cmd_len = MAX_COMMAND_SIZE; + + vhost = (struct vhba_host *)shost->hostdata; + memset(vhost, 0, sizeof(*vhost)); + + vhost->shost = shost; + vhost->num_devices = 0; + spin_lock_init(&vhost->dev_lock); + spin_lock_init(&vhost->cmd_lock); + INIT_WORK(&vhost->scan_devices, vhba_scan_devices); + vhost->cmd_next = 0; + for (i = 0; i < vhost->shost->can_queue; i++) { + vhost->commands[i].status = VHBA_REQ_FREE; + } + + platform_set_drvdata(pdev, vhost); + + if (scsi_add_host(shost, &pdev->dev)) { + scsi_host_put(shost); + return -ENOMEM; + } + + return 0; +} + +static int vhba_remove (struct platform_device *pdev) +{ + struct vhba_host *vhost; + struct Scsi_Host *shost; + + vhost = platform_get_drvdata(pdev); + shost = vhost->shost; + + scsi_remove_host(shost); + scsi_host_put(shost); + + return 0; +} + +static void vhba_release (struct device * dev) +{ + return; +} + +static struct platform_device vhba_platform_device = { + .name = "vhba", + .id = -1, + .dev = { + .release = vhba_release, + }, +}; + +static struct platform_driver vhba_platform_driver = { + .driver = { + .owner = THIS_MODULE, + .name = "vhba", + }, + .probe = vhba_probe, + .remove = vhba_remove, +}; + +static int __init vhba_init (void) +{ + int ret; + + ret = platform_device_register(&vhba_platform_device); + if (ret < 0) { + return ret; + } + + ret = platform_driver_register(&vhba_platform_driver); + if (ret < 0) { + platform_device_unregister(&vhba_platform_device); + return ret; + } + + ret = misc_register(&vhba_miscdev); + if (ret < 0) { + platform_driver_unregister(&vhba_platform_driver); + platform_device_unregister(&vhba_platform_device); + return ret; + } + + return 0; +} + +static void __exit vhba_exit(void) +{ + misc_deregister(&vhba_miscdev); + platform_driver_unregister(&vhba_platform_driver); + platform_device_unregister(&vhba_platform_device); +} + +module_init(vhba_init); +module_exit(vhba_exit); + diff --git a/drivers/tty/Kconfig b/drivers/tty/Kconfig index 9510305..c5af5f0 100644 --- a/drivers/tty/Kconfig +++ b/drivers/tty/Kconfig @@ -75,6 +75,19 @@ config VT_CONSOLE_SLEEP def_bool y depends on VT_CONSOLE && PM_SLEEP +config NR_TTY_DEVICES + int "Maximum tty device number" + depends on VT + range 12 63 + default 63 + ---help--- + This option is used to change the number of tty devices in /dev. + The default value is 63. The lowest number you can set is 12, + 63 is also the upper limit so we don't overrun the serial + consoles. + + If unsure, say 63. + config HW_CONSOLE bool depends on VT && !UML diff --git a/drivers/video/logo/Kconfig b/drivers/video/logo/Kconfig index 0037104..00944ea 100644 --- a/drivers/video/logo/Kconfig +++ b/drivers/video/logo/Kconfig @@ -15,71 +15,138 @@ config FB_LOGO_EXTRA depends on FB=y default y if SPU_BASE +config LOGO_RANDOM + bool "Select random available logo" + default y + help + Enable this option to use any available logo randomly at bootup. + +comment "Available logos" + +config LOGO_PCK_CLUT224 + bool "224-color Zen Kernel/Meditating Tux logo" + default y + config LOGO_LINUX_MONO bool "Standard black and white Linux logo" - default y + default n config LOGO_LINUX_VGA16 bool "Standard 16-color Linux logo" - default y + default n config LOGO_LINUX_CLUT224 bool "Standard 224-color Linux logo" - default y + default n config LOGO_BLACKFIN_VGA16 bool "16-colour Blackfin Processor Linux logo" depends on BLACKFIN - default y + default n config LOGO_BLACKFIN_CLUT224 bool "224-colour Blackfin Processor Linux logo" depends on BLACKFIN - default y + default n + +config LOGO_OLDPCK_CLUT224 + bool "224-color Old Zen Kernel logo" + depends on LOGO + default n + +config LOGO_ARCH_CLUT224 + bool "224-color Arch Linux logo" + depends on LOGO + default n + +config LOGO_GENTOO_CLUT224 + bool "224-color Gentoo Linux logo" + depends on LOGO + default n + +config LOGO_EXHERBO_CLUT224 + bool "224-color Exherbo Linux logo" + depends on LOGO + default n + +config LOGO_SLACKWARE_CLUT224 + bool "224-color Slackware Linux logo" + depends on LOGO + default n + +config LOGO_DEBIAN_CLUT224 + bool "224-color Debian Linux logo" + depends on LOGO + default n + +config LOGO_FEDORASIMPLE_CLUT224 + bool "224-color Fedora Simple Linux logo" + depends on LOGO + default n + +config LOGO_FEDORAGLOSSY_CLUT224 + bool "224-color Fedora Glossy Linux logo" + depends on LOGO + default n + +config LOGO_TITS_CLUT224 + bool "224-color Tits logo" + depends on LOGO + default n + +config LOGO_BSD_CLUT224 + bool "224-color BSD Devil logo" + depends on LOGO + default n + +config LOGO_FBSD_CLUT224 + bool "224-color FreeBSD logo" + depends on LOGO + default n config LOGO_DEC_CLUT224 bool "224-color Digital Equipment Corporation Linux logo" depends on MACH_DECSTATION || ALPHA - default y + default n config LOGO_MAC_CLUT224 bool "224-color Macintosh Linux logo" depends on MAC - default y + default n config LOGO_PARISC_CLUT224 bool "224-color PA-RISC Linux logo" depends on PARISC - default y + default n config LOGO_SGI_CLUT224 bool "224-color SGI Linux logo" depends on SGI_IP22 || SGI_IP27 || SGI_IP32 - default y + default n config LOGO_SUN_CLUT224 bool "224-color Sun Linux logo" depends on SPARC - default y + default n config LOGO_SUPERH_MONO bool "Black and white SuperH Linux logo" depends on SUPERH - default y + default n config LOGO_SUPERH_VGA16 bool "16-color SuperH Linux logo" depends on SUPERH - default y + default n config LOGO_SUPERH_CLUT224 bool "224-color SuperH Linux logo" depends on SUPERH - default y + default n config LOGO_M32R_CLUT224 bool "224-color M32R Linux logo" depends on M32R - default y + default n endif # LOGO diff --git a/drivers/video/logo/Makefile b/drivers/video/logo/Makefile index 3b43781..84d4f11 100644 --- a/drivers/video/logo/Makefile +++ b/drivers/video/logo/Makefile @@ -6,6 +6,18 @@ obj-$(CONFIG_LOGO_LINUX_VGA16) += logo_linux_vga16.o obj-$(CONFIG_LOGO_LINUX_CLUT224) += logo_linux_clut224.o obj-$(CONFIG_LOGO_BLACKFIN_CLUT224) += logo_blackfin_clut224.o obj-$(CONFIG_LOGO_BLACKFIN_VGA16) += logo_blackfin_vga16.o +obj-$(CONFIG_LOGO_PCK_CLUT224) += logo_zen_clut224.o +obj-$(CONFIG_LOGO_OLDPCK_CLUT224) += logo_oldzen_clut224.o +obj-$(CONFIG_LOGO_ARCH_CLUT224) += logo_arch_clut224.o +obj-$(CONFIG_LOGO_GENTOO_CLUT224) += logo_gentoo_clut224.o +obj-$(CONFIG_LOGO_EXHERBO_CLUT224) += logo_exherbo_clut224.o +obj-$(CONFIG_LOGO_SLACKWARE_CLUT224) += logo_slackware_clut224.o +obj-$(CONFIG_LOGO_DEBIAN_CLUT224) += logo_debian_clut224.o +obj-$(CONFIG_LOGO_FEDORASIMPLE_CLUT224) += logo_fedorasimple_clut224.o +obj-$(CONFIG_LOGO_FEDORAGLOSSY_CLUT224) += logo_fedoraglossy_clut224.o +obj-$(CONFIG_LOGO_TITS_CLUT224) += logo_tits_clut224.o +obj-$(CONFIG_LOGO_BSD_CLUT224) += logo_bsd_clut224.o +obj-$(CONFIG_LOGO_FBSD_CLUT224) += logo_fbsd_clut224.o obj-$(CONFIG_LOGO_DEC_CLUT224) += logo_dec_clut224.o obj-$(CONFIG_LOGO_MAC_CLUT224) += logo_mac_clut224.o obj-$(CONFIG_LOGO_PARISC_CLUT224) += logo_parisc_clut224.o diff --git a/drivers/video/logo/logo.c b/drivers/video/logo/logo.c index 4d50bfd..d3a1335 100644 --- a/drivers/video/logo/logo.c +++ b/drivers/video/logo/logo.c @@ -1,26 +1,127 @@ /* - * Linux logo to be displayed on boot - * - * Copyright (C) 1996 Larry Ewing (lewing@isc.tamu.edu) - * Copyright (C) 1996,1998 Jakub Jelinek (jj@sunsite.mff.cuni.cz) - * Copyright (C) 2001 Greg Banks - * Copyright (C) 2001 Jan-Benedict Glaw - * Copyright (C) 2003 Geert Uytterhoeven - */ +* Linux logo to be displayed on boot +* +* Copyright (C) 1996 Larry Ewing (lewing@isc.tamu.edu) +* Copyright (C) 1996,1998 Jakub Jelinek (jj@sunsite.mff.cuni.cz) +* Copyright (C) 2001 Greg Banks +* Copyright (C) 2001 Jan-Benedict Glaw +* Copyright (C) 2003 Geert Uytterhoeven +*/ #include #include #include +#ifdef CONFIG_LOGO_RANDOM +#include +#endif + #ifdef CONFIG_M68K #include #endif + static bool nologo; module_param(nologo, bool, 0); MODULE_PARM_DESC(nologo, "Disables startup logo"); +/* Monochromatic logos */ +static const struct linux_logo *logo_mono[] = { +#ifdef CONFIG_LOGO_LINUX_MONO + &logo_linux_mono, /* Generic Linux logo */ +#endif +#ifdef CONFIG_LOGO_SUPERH_MONO + &logo_superh_mono, /* SuperH Linux logo */ +#endif +}; + +/* 16-colour logos */ +static const struct linux_logo *logo_vga16[] = { +#ifdef CONFIG_LOGO_LINUX_VGA16 + &logo_linux_vga16, /* Generic Linux logo */ +#endif +#ifdef CONFIG_LOGO_BLACKFIN_VGA16 + &logo_blackfin_vga16, /* Blackfin processor logo */ +#endif +#ifdef CONFIG_LOGO_SUPERH_VGA16 + &logo_superh_vga16, /* SuperH Linux logo */ +#endif +}; + +/* 224-colour logos */ +static const struct linux_logo *logo_clut224[] = { +#ifdef CONFIG_LOGO_LINUX_CLUT224 + &logo_linux_clut224, /* Generic Linux logo */ +#endif +#ifdef CONFIG_LOGO_BLACKFIN_CLUT224 + &logo_blackfin_clut224, /* Blackfin Linux logo */ +#endif +#ifdef CONFIG_LOGO_DEC_CLUT224 + &logo_dec_clut224, /* DEC Linux logo on MIPS/MIPS64 or ALPHA */ +#endif +#ifdef CONFIG_LOGO_MAC_CLUT224 + &logo_mac_clut224, /* Macintosh Linux logo on m68k */ +#endif +#ifdef CONFIG_LOGO_PARISC_CLUT224 + &logo_parisc_clut224, /* PA-RISC Linux logo */ +#endif +#ifdef CONFIG_LOGO_SGI_CLUT224 + &logo_sgi_clut224, /* SGI Linux logo on MIPS/MIPS64 */ +#endif +#ifdef CONFIG_LOGO_SUN_CLUT224 + &logo_sun_clut224, /* Sun Linux logo */ +#endif +#ifdef CONFIG_LOGO_SUPERH_CLUT224 + &logo_superh_clut224, /* SuperH Linux logo */ +#endif +#ifdef CONFIG_LOGO_M32R_CLUT224 + &logo_m32r_clut224, /* M32R Linux logo */ +#endif +#ifdef CONFIG_LOGO_PCK_CLUT224 + &logo_zen_clut224, /* Zen-Kernel logo */ +#endif +#ifdef CONFIG_LOGO_OLDPCK_CLUT224 + &logo_oldzen_clut224, /* Old Zen-Kernel logo */ +#endif +#ifdef CONFIG_LOGO_ARCH_CLUT224 + &logo_arch_clut224, /* Arch Linux logo */ +#endif +#ifdef CONFIG_LOGO_GENTOO_CLUT224 + &logo_gentoo_clut224, /* Gentoo Linux logo */ +#endif +#ifdef CONFIG_LOGO_EXHERBO_CLUT224 + &logo_exherbo_clut224, /* Exherbo Linux logo */ +#endif +#ifdef CONFIG_LOGO_SLACKWARE_CLUT224 + &logo_slackware_clut224, /* Slackware Linux logo */ +#endif +#ifdef CONFIG_LOGO_DEBIAN_CLUT224 + &logo_debian_clut224, /* Debian Linux logo */ +#endif +#ifdef CONFIG_LOGO_FEDORASIMPLE_CLUT224 + &logo_fedorasimple_clut224, /* Fedora Simple logo */ +#endif +#ifdef CONFIG_LOGO_FEDORAGLOSSY_CLUT224 + &logo_fedoraglossy_clut224, /* Fedora Glossy logo */ +#endif +#ifdef CONFIG_LOGO_TITS_CLUT224 + &logo_tits_clut224, /* Tits logo */ +#endif +#ifdef CONFIG_LOGO_BSD_CLUT224 + &logo_bsd_clut224, /* BSD logo */ +#endif +#ifdef CONFIG_LOGO_FBSD_CLUT224 + &logo_fbsd_clut224, /* Free BSD logo */ +#endif +}; + +#ifdef CONFIG_LOGO_RANDOM +#define LOGO_INDEX(s) (get_random_int() % s) +#else +#define LOGO_INDEX(s) (s - 1) +#endif + /* * Logos are located in the initdata, and will be freed in kernel_init. * Use late_init to mark the logos as freed to prevent any further use. @@ -43,75 +144,30 @@ late_initcall_sync(fb_logo_late_init); const struct linux_logo * __ref fb_find_logo(int depth) { const struct linux_logo *logo = NULL; + const struct linux_logo **array = NULL; + unsigned int size; if (nologo || logos_freed) return NULL; + /* Select logo array */ if (depth >= 1) { -#ifdef CONFIG_LOGO_LINUX_MONO - /* Generic Linux logo */ - logo = &logo_linux_mono; -#endif -#ifdef CONFIG_LOGO_SUPERH_MONO - /* SuperH Linux logo */ - logo = &logo_superh_mono; -#endif + array = logo_mono; + size = ARRAY_SIZE(logo_mono); } - if (depth >= 4) { -#ifdef CONFIG_LOGO_LINUX_VGA16 - /* Generic Linux logo */ - logo = &logo_linux_vga16; -#endif -#ifdef CONFIG_LOGO_BLACKFIN_VGA16 - /* Blackfin processor logo */ - logo = &logo_blackfin_vga16; -#endif -#ifdef CONFIG_LOGO_SUPERH_VGA16 - /* SuperH Linux logo */ - logo = &logo_superh_vga16; -#endif + array = logo_vga16; + size = ARRAY_SIZE(logo_vga16); } - if (depth >= 8) { -#ifdef CONFIG_LOGO_LINUX_CLUT224 - /* Generic Linux logo */ - logo = &logo_linux_clut224; -#endif -#ifdef CONFIG_LOGO_BLACKFIN_CLUT224 - /* Blackfin Linux logo */ - logo = &logo_blackfin_clut224; -#endif -#ifdef CONFIG_LOGO_DEC_CLUT224 - /* DEC Linux logo on MIPS/MIPS64 or ALPHA */ - logo = &logo_dec_clut224; -#endif -#ifdef CONFIG_LOGO_MAC_CLUT224 - /* Macintosh Linux logo on m68k */ - if (MACH_IS_MAC) - logo = &logo_mac_clut224; -#endif -#ifdef CONFIG_LOGO_PARISC_CLUT224 - /* PA-RISC Linux logo */ - logo = &logo_parisc_clut224; -#endif -#ifdef CONFIG_LOGO_SGI_CLUT224 - /* SGI Linux logo on MIPS/MIPS64 */ - logo = &logo_sgi_clut224; -#endif -#ifdef CONFIG_LOGO_SUN_CLUT224 - /* Sun Linux logo */ - logo = &logo_sun_clut224; -#endif -#ifdef CONFIG_LOGO_SUPERH_CLUT224 - /* SuperH Linux logo */ - logo = &logo_superh_clut224; -#endif -#ifdef CONFIG_LOGO_M32R_CLUT224 - /* M32R Linux logo */ - logo = &logo_m32r_clut224; -#endif + array = logo_clut224; + size = ARRAY_SIZE(logo_clut224); } + + /* We've got some logos to display */ + if (array && size) + logo = array[LOGO_INDEX(size)]; + return logo; } EXPORT_SYMBOL_GPL(fb_find_logo); diff --git b/drivers/video/logo/logo_arch_clut224.ppm b/drivers/video/logo/logo_arch_clut224.ppm new file mode 100644 index 0000000..e4d8daa --- /dev/null +++ b/drivers/video/logo/logo_arch_clut224.ppm @@ -0,0 +1,43204 @@ +P3 +# CREATOR: GIMP PNM Filter Version 1.1 +120 120 +255 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +10 +110 +160 +33 +122 +166 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +27 +151 +213 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +4 +45 +68 +13 +147 +209 +13 +147 +209 +17 +73 +101 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +24 +137 +199 +13 +147 +209 +13 +147 +209 +54 +155 +212 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +7 +10 +13 +147 +209 +13 +147 +209 +13 +147 +209 +48 +164 +219 +3 +23 +31 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +33 +133 +182 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +150 +212 +40 +160 +215 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +65 +166 +216 +5 +11 +14 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +3 +76 +109 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +65 +166 +216 +54 +136 +181 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +42 +161 +216 +48 +164 +219 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +9 +18 +24 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +48 +164 +219 +40 +90 +118 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +37 +132 +189 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +65 +166 +216 +48 +164 +219 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +48 +164 +219 +48 +164 +219 +11 +35 +49 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +25 +87 +120 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +27 +151 +213 +62 +163 +214 +62 +163 +214 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +65 +166 +216 +48 +164 +219 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +17 +148 +210 +17 +148 +210 +17 +148 +210 +17 +148 +210 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +62 +163 +214 +48 +164 +219 +36 +86 +115 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +10 +110 +160 +21 +149 +211 +21 +149 +211 +17 +148 +210 +17 +148 +210 +17 +148 +210 +17 +148 +210 +17 +148 +210 +13 +147 +209 +13 +147 +209 +13 +147 +209 +49 +151 +208 +48 +164 +219 +65 +166 +216 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +17 +148 +210 +24 +150 +212 +21 +149 +211 +13 +147 +209 +13 +147 +209 +21 +149 +211 +17 +148 +210 +17 +148 +210 +17 +148 +210 +17 +148 +210 +17 +148 +210 +13 +147 +209 +65 +166 +216 +65 +166 +216 +11 +35 +49 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +74 +106 +31 +155 +211 +24 +150 +212 +24 +150 +212 +27 +151 +213 +17 +148 +210 +21 +149 +211 +13 +147 +209 +13 +147 +209 +49 +151 +208 +21 +149 +211 +17 +148 +210 +17 +148 +210 +48 +164 +219 +65 +166 +216 +62 +163 +214 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +24 +150 +212 +17 +148 +210 +31 +155 +211 +17 +148 +210 +17 +148 +210 +2 +145 +206 +31 +155 +211 +31 +155 +211 +27 +151 +213 +21 +149 +211 +49 +151 +208 +21 +149 +211 +21 +149 +211 +49 +151 +208 +65 +166 +216 +65 +166 +216 +5 +18 +28 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +27 +39 +27 +151 +213 +27 +151 +213 +49 +151 +208 +24 +150 +212 +24 +150 +212 +31 +155 +211 +24 +150 +212 +21 +149 +211 +24 +150 +212 +27 +151 +213 +27 +151 +213 +2 +145 +206 +21 +149 +211 +21 +149 +211 +72 +171 +221 +62 +163 +214 +48 +154 +203 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +143 +204 +31 +155 +211 +31 +155 +211 +24 +150 +212 +31 +155 +211 +27 +151 +213 +24 +150 +212 +24 +150 +212 +31 +155 +211 +31 +155 +211 +21 +149 +211 +31 +155 +211 +31 +155 +211 +21 +149 +211 +27 +151 +213 +62 +163 +214 +67 +167 +217 +67 +167 +217 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +31 +155 +211 +31 +155 +211 +31 +155 +211 +31 +155 +211 +24 +150 +212 +24 +150 +212 +24 +150 +212 +21 +149 +211 +31 +155 +211 +29 +152 +214 +21 +149 +211 +17 +148 +210 +31 +155 +211 +17 +148 +210 +24 +150 +212 +49 +151 +208 +67 +167 +217 +67 +167 +217 +49 +132 +177 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +29 +108 +153 +51 +153 +210 +51 +153 +210 +31 +155 +211 +31 +155 +211 +24 +150 +212 +24 +150 +212 +31 +155 +211 +31 +155 +211 +24 +150 +212 +24 +150 +212 +27 +151 +213 +27 +151 +213 +49 +151 +208 +21 +149 +211 +21 +149 +211 +27 +151 +213 +67 +167 +217 +67 +167 +217 +67 +167 +217 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +49 +151 +208 +31 +155 +211 +51 +153 +210 +29 +152 +214 +32 +153 +215 +32 +153 +215 +29 +152 +214 +31 +155 +211 +24 +150 +212 +24 +150 +212 +24 +150 +212 +31 +155 +211 +24 +150 +212 +31 +155 +211 +31 +155 +211 +27 +151 +213 +24 +150 +212 +62 +163 +214 +68 +168 +218 +68 +168 +218 +40 +90 +118 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +71 +103 +33 +156 +212 +31 +155 +211 +31 +155 +211 +31 +155 +211 +49 +151 +208 +29 +152 +214 +31 +155 +211 +31 +155 +211 +32 +153 +215 +24 +150 +212 +24 +150 +212 +31 +155 +211 +31 +155 +211 +24 +150 +212 +31 +155 +211 +31 +155 +211 +24 +150 +212 +29 +152 +214 +78 +167 +212 +68 +168 +218 +65 +166 +216 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +31 +155 +211 +31 +155 +211 +33 +156 +212 +32 +153 +215 +32 +153 +215 +51 +153 +210 +51 +153 +210 +31 +155 +211 +49 +151 +208 +29 +152 +214 +29 +152 +214 +31 +155 +211 +32 +153 +215 +31 +155 +211 +31 +155 +211 +24 +150 +212 +24 +150 +212 +24 +150 +212 +31 +155 +211 +68 +168 +218 +69 +169 +219 +71 +170 +220 +16 +56 +73 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +11 +35 +49 +33 +156 +212 +33 +156 +212 +53 +154 +211 +53 +154 +211 +31 +155 +211 +31 +155 +211 +31 +155 +211 +51 +153 +210 +31 +155 +211 +31 +155 +211 +31 +155 +211 +51 +153 +210 +51 +153 +210 +29 +152 +214 +51 +153 +210 +31 +155 +211 +31 +155 +211 +24 +150 +212 +31 +155 +211 +31 +155 +211 +69 +169 +219 +69 +169 +219 +65 +166 +216 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +49 +151 +208 +32 +153 +215 +33 +156 +212 +32 +153 +215 +54 +155 +212 +31 +155 +211 +32 +153 +215 +32 +153 +215 +33 +156 +212 +31 +155 +211 +32 +153 +215 +29 +152 +214 +51 +153 +210 +32 +153 +215 +31 +155 +211 +31 +155 +211 +29 +152 +214 +31 +155 +211 +51 +153 +210 +31 +155 +211 +31 +155 +211 +68 +168 +218 +82 +170 +215 +69 +169 +219 +12 +30 +39 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +18 +23 +54 +155 +212 +54 +155 +212 +33 +156 +212 +33 +156 +212 +33 +156 +212 +33 +156 +212 +38 +159 +214 +53 +154 +211 +53 +154 +211 +33 +156 +212 +31 +155 +211 +31 +155 +211 +33 +156 +212 +32 +153 +215 +32 +153 +215 +31 +155 +211 +31 +155 +211 +51 +153 +210 +31 +155 +211 +31 +155 +211 +31 +155 +211 +44 +162 +217 +82 +170 +215 +71 +170 +220 +62 +163 +214 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +41 +147 +203 +42 +161 +216 +38 +159 +214 +54 +155 +212 +38 +159 +214 +38 +159 +214 +32 +153 +215 +33 +156 +212 +32 +153 +215 +32 +153 +215 +54 +155 +212 +54 +155 +212 +32 +153 +215 +31 +155 +211 +31 +155 +211 +32 +153 +215 +32 +153 +215 +33 +156 +212 +31 +155 +211 +49 +151 +208 +31 +155 +211 +49 +151 +208 +32 +153 +215 +72 +171 +221 +71 +170 +220 +71 +170 +220 +8 +27 +35 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +9 +12 +42 +161 +216 +42 +161 +216 +54 +155 +212 +54 +155 +212 +42 +161 +216 +54 +155 +212 +54 +155 +212 +38 +159 +214 +38 +159 +214 +38 +159 +214 +54 +155 +212 +38 +159 +214 +33 +156 +212 +33 +156 +212 +33 +156 +212 +53 +154 +211 +32 +153 +215 +32 +153 +215 +31 +155 +211 +33 +156 +212 +32 +153 +215 +32 +153 +215 +32 +153 +215 +62 +163 +214 +82 +170 +215 +72 +171 +221 +55 +159 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +42 +139 +189 +44 +162 +217 +44 +162 +217 +54 +155 +212 +42 +161 +216 +42 +161 +216 +54 +155 +212 +54 +155 +212 +42 +161 +216 +38 +159 +214 +54 +155 +212 +54 +155 +212 +40 +160 +215 +54 +155 +212 +33 +156 +212 +38 +159 +214 +54 +155 +212 +54 +155 +212 +33 +156 +212 +33 +156 +212 +53 +154 +211 +33 +156 +212 +31 +155 +211 +31 +155 +211 +54 +155 +212 +72 +171 +221 +72 +171 +221 +71 +170 +220 +5 +11 +14 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +44 +162 +217 +55 +159 +209 +54 +155 +212 +54 +155 +212 +54 +155 +212 +54 +155 +212 +54 +155 +212 +42 +161 +216 +42 +161 +216 +42 +161 +216 +42 +161 +216 +42 +161 +216 +54 +155 +212 +38 +159 +214 +38 +159 +214 +38 +159 +214 +38 +159 +214 +38 +159 +214 +33 +156 +212 +54 +155 +212 +54 +155 +212 +33 +156 +212 +53 +154 +211 +32 +153 +215 +33 +156 +212 +68 +168 +218 +74 +172 +223 +72 +171 +221 +48 +154 +203 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +51 +74 +40 +160 +215 +55 +159 +209 +42 +161 +216 +42 +161 +216 +54 +155 +212 +40 +160 +215 +62 +163 +214 +44 +162 +217 +42 +161 +216 +54 +155 +212 +54 +155 +212 +42 +161 +216 +54 +155 +212 +54 +155 +212 +54 +155 +212 +54 +155 +212 +54 +155 +212 +38 +159 +214 +54 +155 +212 +40 +160 +215 +32 +153 +215 +54 +155 +212 +33 +156 +212 +38 +159 +214 +33 +156 +212 +31 +155 +211 +83 +172 +217 +74 +172 +223 +82 +170 +215 +0 +7 +10 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +24 +70 +92 +48 +164 +219 +62 +163 +214 +54 +155 +212 +54 +155 +212 +55 +159 +209 +46 +163 +218 +54 +155 +212 +46 +163 +218 +55 +159 +209 +54 +155 +212 +54 +155 +212 +42 +161 +216 +42 +161 +216 +42 +161 +216 +42 +161 +216 +42 +161 +216 +54 +155 +212 +38 +159 +214 +54 +155 +212 +38 +159 +214 +40 +160 +215 +32 +153 +215 +54 +155 +212 +54 +155 +212 +33 +156 +212 +72 +171 +221 +74 +172 +223 +74 +172 +223 +53 +145 +195 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +23 +77 +105 +44 +162 +217 +48 +164 +219 +44 +162 +217 +54 +155 +212 +42 +161 +216 +46 +163 +218 +62 +163 +214 +40 +160 +215 +55 +159 +209 +44 +162 +217 +40 +160 +215 +33 +156 +212 +54 +155 +212 +54 +155 +212 +42 +161 +216 +54 +155 +212 +42 +161 +216 +42 +161 +216 +42 +161 +216 +54 +155 +212 +38 +159 +214 +38 +159 +214 +40 +160 +215 +40 +160 +215 +33 +156 +212 +74 +172 +223 +83 +172 +217 +74 +172 +223 +0 +7 +10 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +36 +86 +115 +48 +164 +219 +44 +162 +217 +44 +162 +217 +54 +155 +212 +62 +163 +214 +46 +163 +218 +40 +160 +215 +42 +161 +216 +44 +162 +217 +46 +163 +218 +55 +159 +209 +44 +162 +217 +33 +156 +212 +55 +159 +209 +42 +161 +216 +54 +155 +212 +42 +161 +216 +42 +161 +216 +54 +155 +212 +42 +161 +216 +42 +161 +216 +38 +159 +214 +54 +155 +212 +38 +159 +214 +74 +172 +223 +74 +172 +223 +86 +174 +219 +54 +136 +181 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +23 +88 +115 +5 +18 +28 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +24 +70 +92 +62 +163 +214 +54 +155 +212 +62 +163 +214 +44 +162 +217 +54 +155 +212 +48 +164 +219 +48 +164 +219 +54 +155 +212 +42 +161 +216 +48 +164 +219 +42 +161 +216 +54 +155 +212 +40 +160 +215 +54 +155 +212 +40 +160 +215 +40 +160 +215 +54 +155 +212 +42 +161 +216 +42 +161 +216 +42 +161 +216 +42 +161 +216 +42 +161 +216 +54 +155 +212 +42 +161 +216 +74 +172 +223 +86 +174 +219 +83 +172 +217 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +62 +163 +214 +67 +167 +217 +36 +86 +115 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +16 +56 +73 +62 +163 +214 +46 +163 +218 +62 +163 +214 +44 +162 +217 +46 +163 +218 +44 +162 +217 +62 +163 +214 +44 +162 +217 +46 +163 +218 +54 +155 +212 +54 +155 +212 +42 +161 +216 +55 +159 +209 +54 +155 +212 +55 +159 +209 +54 +155 +212 +55 +159 +209 +54 +155 +212 +42 +161 +216 +54 +155 +212 +42 +161 +216 +54 +155 +212 +42 +161 +216 +74 +172 +223 +86 +174 +219 +86 +174 +219 +57 +132 +172 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +23 +88 +115 +62 +163 +214 +48 +164 +219 +48 +164 +219 +42 +151 +200 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +18 +42 +55 +44 +162 +217 +62 +163 +214 +48 +164 +219 +62 +163 +214 +54 +155 +212 +44 +162 +217 +65 +166 +216 +46 +163 +218 +48 +164 +219 +44 +162 +217 +62 +163 +214 +42 +161 +216 +62 +163 +214 +46 +163 +218 +44 +162 +217 +54 +155 +212 +55 +159 +209 +54 +155 +212 +40 +160 +215 +40 +160 +215 +54 +155 +212 +42 +161 +216 +55 +159 +209 +86 +174 +219 +86 +174 +219 +85 +173 +218 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +62 +163 +214 +62 +163 +214 +62 +163 +214 +48 +164 +219 +60 +162 +212 +62 +163 +214 +16 +56 +73 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +16 +23 +62 +163 +214 +44 +162 +217 +62 +163 +214 +46 +163 +218 +48 +164 +219 +62 +163 +214 +46 +163 +218 +62 +163 +214 +62 +163 +214 +44 +162 +217 +67 +167 +217 +44 +162 +217 +54 +155 +212 +54 +155 +212 +42 +161 +216 +48 +164 +219 +54 +155 +212 +54 +155 +212 +40 +160 +215 +42 +161 +216 +62 +163 +214 +40 +160 +215 +74 +172 +223 +90 +177 +222 +87 +175 +220 +28 +73 +96 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +19 +48 +67 +48 +164 +219 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +48 +164 +219 +60 +162 +212 +62 +163 +214 +8 +27 +35 +0 +2 +0 +0 +2 +0 +0 +1 +4 +54 +155 +212 +67 +167 +217 +68 +168 +218 +65 +166 +216 +62 +163 +214 +62 +163 +214 +65 +166 +216 +62 +163 +214 +62 +163 +214 +44 +162 +217 +48 +164 +219 +54 +155 +212 +62 +163 +214 +48 +164 +219 +62 +163 +214 +54 +155 +212 +42 +161 +216 +54 +155 +212 +46 +163 +218 +42 +161 +216 +42 +161 +216 +54 +155 +212 +87 +175 +220 +87 +175 +220 +85 +173 +218 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +67 +167 +217 +65 +166 +216 +67 +167 +217 +65 +166 +216 +62 +163 +214 +65 +166 +216 +48 +164 +219 +65 +166 +216 +62 +163 +214 +65 +166 +216 +53 +145 +195 +0 +7 +10 +0 +2 +0 +0 +4 +7 +58 +151 +195 +54 +155 +212 +48 +164 +219 +69 +169 +219 +48 +164 +219 +46 +163 +218 +48 +164 +219 +46 +163 +218 +54 +155 +212 +62 +163 +214 +54 +155 +212 +44 +162 +217 +42 +161 +216 +62 +163 +214 +48 +164 +219 +44 +162 +217 +54 +155 +212 +44 +162 +217 +54 +155 +212 +38 +159 +214 +48 +164 +219 +83 +172 +217 +90 +177 +222 +90 +177 +222 +30 +93 +120 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +22 +45 +59 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +72 +171 +221 +65 +166 +216 +48 +164 +219 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +33 +133 +182 +0 +4 +7 +0 +1 +4 +34 +104 +137 +62 +163 +214 +62 +163 +214 +62 +163 +214 +62 +163 +214 +67 +167 +217 +69 +169 +219 +62 +163 +214 +62 +163 +214 +48 +164 +219 +62 +163 +214 +62 +163 +214 +62 +163 +214 +48 +164 +219 +65 +166 +216 +42 +161 +216 +48 +164 +219 +48 +164 +219 +44 +162 +217 +55 +159 +209 +38 +159 +214 +88 +176 +221 +88 +176 +221 +85 +173 +218 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +65 +166 +216 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +67 +167 +217 +62 +163 +214 +65 +166 +216 +62 +163 +214 +48 +164 +219 +68 +168 +218 +36 +115 +153 +0 +4 +7 +17 +35 +44 +62 +163 +214 +65 +166 +216 +62 +163 +214 +48 +164 +219 +48 +164 +219 +62 +163 +214 +67 +167 +217 +68 +168 +218 +62 +163 +214 +65 +166 +216 +62 +163 +214 +44 +162 +217 +65 +166 +216 +62 +163 +214 +62 +163 +214 +48 +164 +219 +62 +163 +214 +44 +162 +217 +54 +155 +212 +72 +171 +221 +88 +176 +221 +88 +176 +221 +40 +90 +118 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +26 +49 +63 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +67 +167 +217 +48 +164 +219 +48 +164 +219 +65 +166 +216 +62 +163 +214 +31 +134 +178 +5 +11 +14 +48 +154 +203 +60 +162 +212 +48 +164 +219 +62 +163 +214 +48 +164 +219 +68 +168 +218 +48 +164 +219 +62 +163 +214 +62 +163 +214 +68 +168 +218 +62 +163 +214 +48 +164 +219 +65 +166 +216 +54 +155 +212 +62 +163 +214 +44 +162 +217 +48 +164 +219 +62 +163 +214 +42 +161 +216 +90 +177 +222 +88 +176 +221 +85 +173 +218 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +62 +163 +214 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +67 +167 +217 +65 +166 +216 +62 +163 +214 +62 +163 +214 +48 +154 +203 +53 +127 +166 +48 +164 +219 +65 +166 +216 +62 +163 +214 +48 +164 +219 +62 +163 +214 +54 +155 +212 +69 +169 +219 +48 +164 +219 +67 +167 +217 +48 +164 +219 +62 +163 +214 +46 +163 +218 +62 +163 +214 +62 +163 +214 +46 +163 +218 +46 +163 +218 +46 +163 +218 +74 +172 +223 +90 +177 +222 +90 +177 +222 +25 +99 +131 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +22 +45 +59 +78 +167 +212 +78 +167 +212 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +48 +164 +219 +48 +164 +219 +62 +163 +214 +48 +164 +219 +62 +163 +214 +65 +166 +216 +48 +164 +219 +48 +164 +219 +60 +162 +212 +48 +164 +219 +64 +165 +215 +62 +163 +214 +71 +170 +220 +48 +164 +219 +62 +163 +214 +48 +164 +219 +62 +163 +214 +62 +163 +214 +44 +162 +217 +62 +163 +214 +62 +163 +214 +90 +177 +222 +90 +177 +222 +74 +172 +223 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +78 +167 +212 +71 +170 +220 +69 +169 +219 +69 +169 +219 +69 +169 +219 +69 +169 +219 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +68 +168 +218 +65 +166 +216 +62 +163 +214 +65 +166 +216 +65 +166 +216 +65 +166 +216 +48 +164 +219 +65 +166 +216 +48 +164 +219 +48 +164 +219 +62 +163 +214 +48 +164 +219 +62 +163 +214 +48 +164 +219 +54 +155 +212 +48 +164 +219 +69 +169 +219 +62 +163 +214 +78 +167 +212 +102 +181 +221 +90 +177 +222 +34 +104 +137 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +19 +48 +67 +71 +170 +220 +71 +170 +220 +69 +169 +219 +78 +167 +212 +71 +170 +220 +78 +167 +212 +71 +170 +220 +69 +169 +219 +69 +169 +219 +71 +170 +220 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +68 +168 +218 +65 +166 +216 +62 +163 +214 +65 +166 +216 +48 +164 +219 +48 +164 +219 +48 +164 +219 +48 +164 +219 +62 +163 +214 +48 +164 +219 +62 +163 +214 +62 +163 +214 +62 +163 +214 +65 +166 +216 +48 +164 +219 +62 +163 +214 +90 +177 +222 +88 +176 +221 +87 +175 +220 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +69 +169 +219 +82 +170 +215 +82 +170 +215 +69 +169 +219 +71 +170 +220 +82 +170 +215 +71 +170 +220 +78 +167 +212 +78 +167 +212 +71 +170 +220 +69 +169 +219 +71 +170 +220 +78 +167 +212 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +68 +168 +218 +62 +163 +214 +62 +163 +214 +65 +166 +216 +62 +163 +214 +65 +166 +216 +48 +164 +219 +64 +165 +215 +62 +163 +214 +48 +164 +219 +48 +164 +219 +62 +163 +214 +62 +163 +214 +65 +166 +216 +91 +178 +224 +91 +178 +224 +21 +105 +143 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +22 +61 +78 +74 +172 +223 +82 +170 +215 +82 +170 +215 +72 +171 +221 +72 +171 +221 +71 +170 +220 +71 +170 +220 +82 +170 +215 +69 +169 +219 +71 +170 +220 +69 +169 +219 +69 +169 +219 +69 +169 +219 +69 +169 +219 +69 +169 +219 +69 +169 +219 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +62 +163 +214 +65 +166 +216 +62 +163 +214 +65 +166 +216 +48 +164 +219 +48 +164 +219 +65 +166 +216 +65 +166 +216 +60 +162 +212 +60 +162 +212 +48 +164 +219 +91 +178 +224 +91 +178 +224 +88 +176 +221 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +4 +82 +170 +215 +74 +172 +223 +83 +172 +217 +83 +172 +217 +83 +172 +217 +82 +170 +215 +72 +171 +221 +72 +171 +221 +82 +170 +215 +71 +170 +220 +69 +169 +219 +71 +170 +220 +71 +170 +220 +82 +170 +215 +69 +169 +219 +71 +170 +220 +69 +169 +219 +71 +170 +220 +69 +169 +219 +78 +167 +212 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +67 +167 +217 +65 +166 +216 +62 +163 +214 +62 +163 +214 +65 +166 +216 +48 +164 +219 +67 +167 +217 +65 +166 +216 +62 +163 +214 +90 +177 +222 +91 +178 +224 +48 +115 +149 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +45 +104 +131 +74 +172 +223 +83 +172 +217 +74 +172 +223 +83 +172 +217 +74 +172 +223 +83 +172 +217 +83 +172 +217 +83 +172 +217 +82 +170 +215 +82 +170 +215 +82 +170 +215 +82 +170 +215 +82 +170 +215 +71 +170 +220 +69 +169 +219 +71 +170 +220 +71 +170 +220 +71 +170 +220 +69 +169 +219 +69 +169 +219 +71 +170 +220 +69 +169 +219 +78 +167 +212 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +68 +168 +218 +62 +163 +214 +48 +164 +219 +48 +164 +219 +48 +164 +219 +67 +167 +217 +71 +170 +220 +91 +178 +224 +91 +178 +224 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +85 +173 +218 +74 +172 +223 +85 +173 +218 +85 +173 +218 +74 +172 +223 +74 +172 +223 +74 +172 +223 +83 +172 +217 +74 +172 +223 +83 +172 +217 +83 +172 +217 +83 +172 +217 +72 +171 +221 +72 +171 +221 +72 +171 +221 +72 +171 +221 +71 +170 +220 +82 +170 +215 +82 +170 +215 +71 +170 +220 +82 +170 +215 +69 +169 +219 +78 +167 +212 +69 +169 +219 +69 +169 +219 +78 +167 +212 +69 +169 +219 +69 +169 +219 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +68 +168 +218 +48 +164 +219 +62 +163 +214 +62 +163 +214 +90 +177 +222 +91 +178 +224 +50 +145 +189 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +48 +115 +149 +86 +174 +219 +88 +176 +221 +88 +176 +221 +86 +174 +219 +86 +174 +219 +85 +173 +218 +86 +174 +219 +85 +173 +218 +83 +172 +217 +83 +172 +217 +83 +172 +217 +74 +172 +223 +83 +172 +217 +83 +172 +217 +74 +172 +223 +72 +171 +221 +82 +170 +215 +82 +170 +215 +72 +171 +221 +71 +170 +220 +71 +170 +220 +82 +170 +215 +71 +170 +220 +82 +170 +215 +69 +169 +219 +69 +169 +219 +69 +169 +219 +68 +168 +218 +69 +169 +219 +68 +168 +218 +68 +168 +218 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +68 +168 +218 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +62 +163 +214 +102 +181 +221 +91 +178 +224 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +88 +176 +221 +87 +175 +220 +88 +176 +221 +74 +172 +223 +86 +174 +219 +86 +174 +219 +88 +176 +221 +85 +173 +218 +86 +174 +219 +74 +172 +223 +85 +173 +218 +74 +172 +223 +74 +172 +223 +83 +172 +217 +83 +172 +217 +83 +172 +217 +74 +172 +223 +74 +172 +223 +74 +172 +223 +82 +170 +215 +82 +170 +215 +82 +170 +215 +78 +167 +212 +48 +164 +219 +29 +152 +214 +17 +148 +210 +7 +146 +208 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +49 +151 +208 +33 +156 +212 +48 +164 +219 +65 +166 +216 +62 +163 +214 +67 +167 +217 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +88 +176 +221 +95 +181 +227 +50 +145 +189 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +64 +129 +163 +87 +175 +220 +87 +175 +220 +87 +175 +220 +87 +175 +220 +87 +175 +220 +74 +172 +223 +86 +174 +219 +86 +174 +219 +88 +176 +221 +86 +174 +219 +86 +174 +219 +85 +173 +218 +86 +174 +219 +85 +173 +218 +74 +172 +223 +74 +172 +223 +83 +172 +217 +83 +172 +217 +82 +170 +215 +55 +159 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +51 +153 +210 +62 +163 +214 +65 +166 +216 +65 +166 +216 +65 +166 +216 +65 +166 +216 +93 +180 +226 +91 +178 +224 +2 +9 +12 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +88 +176 +221 +88 +176 +221 +87 +175 +220 +87 +175 +220 +87 +175 +220 +90 +177 +222 +88 +176 +221 +74 +172 +223 +74 +172 +223 +86 +174 +219 +74 +172 +223 +86 +174 +219 +86 +174 +219 +74 +172 +223 +85 +173 +218 +74 +172 +223 +71 +170 +220 +24 +150 +212 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +31 +155 +211 +62 +163 +214 +69 +169 +219 +102 +181 +221 +66 +157 +202 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +68 +139 +173 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +87 +175 +220 +87 +175 +220 +87 +175 +220 +87 +175 +220 +91 +178 +224 +87 +175 +220 +74 +172 +223 +86 +174 +219 +86 +174 +219 +29 +152 +214 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +65 +166 +216 +93 +180 +226 +5 +11 +14 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +87 +175 +220 +87 +175 +220 +74 +172 +223 +82 +170 +215 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +62 +163 +214 +55 +159 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +76 +157 +196 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +87 +175 +220 +62 +163 +214 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +150 +212 +48 +164 +219 +5 +18 +28 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +11 +14 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +68 +168 +218 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +48 +164 +219 +62 +163 +214 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +73 +163 +208 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +78 +167 +212 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +67 +167 +217 +0 +7 +10 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +0 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +83 +172 +217 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +42 +151 +200 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +73 +163 +208 +88 +176 +221 +88 +176 +221 +88 +176 +221 +88 +176 +221 +83 +172 +217 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +54 +155 +212 +2 +27 +39 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +13 +16 +88 +176 +221 +88 +176 +221 +88 +176 +221 +86 +174 +219 +21 +149 +211 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +55 +159 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +78 +167 +212 +88 +176 +221 +87 +175 +220 +33 +156 +212 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +14 +38 +51 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +12 +30 +39 +88 +176 +221 +71 +170 +220 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +31 +155 +211 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +102 +181 +221 +17 +148 +210 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +8 +135 +190 +3 +28 +41 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +5 +18 +28 +7 +123 +172 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +19 +48 +67 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +10 +42 +60 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +143 +204 +1 +4 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +4 +16 +137 +192 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +139 +199 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +23 +115 +165 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +60 +88 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +65 +92 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +8 +135 +190 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +10 +110 +160 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +33 +120 +170 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +85 +122 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +6 +88 +126 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +7 +10 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +90 +129 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +18 +28 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +102 +146 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +10 +110 +160 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +129 +183 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +1 +11 +20 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +8 +33 +46 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +139 +199 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +16 +137 +192 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +24 +126 +182 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +4 +14 +21 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +22 +30 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +90 +129 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +11 +14 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +28 +141 +196 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +24 +37 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +4 +45 +68 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +26 +97 +136 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +19 +113 +163 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +4 +14 +21 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +10 +42 +60 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +71 +103 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +6 +31 +44 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +148 +210 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +34 +143 +199 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +60 +88 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +9 +99 +143 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +27 +117 +167 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +10 +69 +97 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +6 +39 +56 +23 +115 +165 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +6 +39 +56 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +85 +122 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +28 +141 +196 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +27 +39 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +126 +182 +0 +4 +7 +0 +4 +7 +18 +92 +131 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +31 +129 +185 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +9 +12 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +17 +102 +146 +0 +2 +0 +0 +2 +0 +2 +9 +12 +19 +113 +163 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +11 +61 +83 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +33 +133 +182 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +27 +117 +167 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +85 +122 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +11 +14 +33 +133 +182 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +11 +14 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +2 +111 +154 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +4 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +137 +199 +5 +11 +14 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +3 +6 +20 +127 +176 +13 +147 +209 +2 +111 +154 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +139 +199 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +118 +167 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +9 +99 +143 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +9 +12 +37 +132 +189 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +27 +39 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +27 +117 +167 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +4 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +0 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +139 +199 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +57 +84 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +60 +87 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +137 +199 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +9 +12 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +102 +146 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +3 +36 +54 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +37 +132 +189 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +96 +140 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +3 +76 +109 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +143 +204 +5 +11 +14 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +129 +183 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +51 +74 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +123 +172 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +4 +45 +68 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +2 +111 +154 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +7 +10 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +17 +102 +146 +0 +24 +37 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +18 +28 +22 +104 +148 +0 +143 +204 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +139 +199 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +28 +141 +196 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +24 +126 +182 +6 +39 +56 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +0 +0 +2 +0 +6 +39 +56 +24 +126 +182 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +6 +39 +56 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +28 +141 +196 +5 +67 +94 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +4 +0 +4 +7 +0 +63 +90 +34 +143 +199 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +17 +102 +146 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +3 +76 +109 +0 +4 +7 +0 +1 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +3 +76 +109 +24 +137 +199 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +21 +149 +211 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +3 +76 +109 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +123 +172 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +7 +10 +33 +120 +170 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +16 +123 +179 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +13 +90 +129 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +90 +129 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +29 +108 +153 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +139 +199 +8 +27 +35 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +27 +39 +34 +143 +199 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +37 +132 +189 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +31 +129 +185 +5 +11 +14 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +9 +12 +37 +132 +189 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +20 +28 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +31 +129 +185 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +23 +115 +165 +0 +1 +4 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +4 +2 +111 +154 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +28 +141 +196 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +13 +90 +129 +0 +1 +4 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +1 +0 +0 +85 +122 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +2 +9 +12 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +127 +181 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +118 +167 +0 +1 +4 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +29 +108 +153 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +137 +199 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +2 +9 +12 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +19 +113 +163 +0 +1 +4 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +10 +110 +160 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +24 +37 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +139 +199 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +118 +167 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +29 +108 +153 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +3 +23 +31 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +129 +183 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +33 +120 +170 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +10 +42 +60 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +118 +167 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +0 +118 +167 +7 +146 +208 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +7 +146 +208 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +85 +122 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +24 +126 +182 +0 +7 +10 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +33 +120 +170 +13 +147 +209 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +96 +140 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +7 +146 +208 +13 +147 +209 +13 +147 +209 +0 +143 +204 +0 +7 +10 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +5 +11 +14 +0 +143 +204 +13 +147 +209 +13 +147 +209 +13 +147 +209 +0 +4 +7 +0 +2 +0 +0 +2 +0 +29 +108 +153 +13 +147 +209 +7 +146 +208 +17 +54 +77 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +17 +54 +77 +7 +146 +208 +13 +147 +209 +0 +118 +167 +0 +2 +0 +0 +4 +7 +7 +146 +208 +17 +102 +146 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +4 +7 +17 +102 +146 +7 +146 +208 +0 +4 +7 +27 +117 +167 +0 +4 +7 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +0 +2 +0 +27 +117 +167 diff --git b/drivers/video/logo/logo_bsd_clut224.ppm b/drivers/video/logo/logo_bsd_clut224.ppm new file mode 100644 index 0000000..408f028 --- /dev/null +++ b/drivers/video/logo/logo_bsd_clut224.ppm @@ -0,0 +1,2403 @@ +P3 +120 120 +255 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +4 0 1 7 1 3 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 28 8 14 +51 24 39 16 5 9 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 9 3 6 39 15 26 61 26 49 58 26 51 +18 7 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 57 17 29 89 29 45 73 26 48 61 26 49 22 7 12 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 6 10 +102 29 42 132 43 63 76 28 47 52 25 42 22 7 12 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 4 0 1 16 5 9 3 1 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 69 20 33 154 39 54 +159 46 62 89 29 45 49 25 40 18 7 11 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 16 5 9 102 29 42 41 13 21 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 97 28 42 185 38 47 145 39 54 +89 29 45 58 26 51 16 5 9 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 4 0 1 132 30 39 148 30 39 +28 8 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 41 13 21 132 30 39 185 38 47 152 40 56 105 35 55 +76 28 47 22 7 12 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 41 13 21 185 38 47 +124 30 42 36 11 20 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 2 4 +117 31 42 185 38 47 185 38 47 152 40 56 105 35 55 76 28 47 +59 25 44 4 0 1 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 4 0 1 121 30 42 +185 38 47 102 29 42 48 19 31 9 3 6 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 123 33 45 +185 38 47 185 38 47 159 38 55 105 35 55 76 28 47 73 26 48 +49 25 40 4 0 1 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 41 13 21 +185 38 47 132 30 39 71 23 37 41 14 25 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 4 0 1 117 31 42 185 38 47 +185 38 47 171 39 51 118 34 52 88 29 45 76 28 47 62 26 49 +52 24 38 18 7 11 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 10 4 6 16 5 9 16 5 9 +18 7 11 18 7 11 16 5 9 10 4 6 16 5 9 7 2 4 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 1 3 +148 30 39 152 32 42 67 22 35 54 21 35 35 12 21 4 0 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 69 20 33 185 38 47 185 38 47 +185 38 47 159 35 49 105 35 55 89 29 45 76 28 47 62 26 49 +60 25 40 42 17 30 3 1 1 0 0 0 0 0 0 0 0 0 +1 0 0 41 13 21 79 27 44 118 34 52 124 30 42 121 30 42 +142 34 47 121 30 42 86 29 44 68 23 35 60 22 36 52 24 38 +42 17 30 30 10 18 16 5 9 9 3 6 9 3 6 3 1 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +148 30 39 185 38 47 71 23 37 50 18 33 49 25 40 35 12 21 +7 1 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 10 4 6 185 38 47 185 38 47 185 38 47 +185 38 47 140 36 52 105 35 55 89 29 45 76 28 47 67 27 47 +56 25 45 52 24 38 48 19 31 39 15 26 30 10 18 31 10 18 +89 29 45 129 39 59 159 38 55 171 39 51 171 39 51 171 39 51 +159 38 55 138 37 53 119 34 51 105 35 55 88 29 45 76 28 47 +76 28 47 62 26 49 57 25 43 49 25 40 48 19 31 31 10 18 +10 4 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 +132 30 39 185 38 47 121 30 42 64 24 39 49 25 40 48 19 31 +10 4 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 102 29 42 185 38 47 185 38 47 185 38 47 +185 38 47 131 38 56 105 35 55 89 29 45 67 27 47 64 25 43 +64 25 43 105 35 55 121 37 56 79 28 45 65 27 44 105 35 55 +159 46 62 159 46 62 159 38 55 171 39 51 171 39 51 144 40 57 +135 40 58 121 37 56 105 35 55 105 35 55 105 35 55 89 29 45 +88 29 45 67 27 47 61 26 49 58 26 51 52 25 42 49 25 40 +48 19 31 35 12 21 7 2 4 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +97 28 42 185 38 47 148 30 39 71 23 37 52 24 38 42 17 30 +35 12 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 36 11 20 171 39 51 185 38 47 185 38 47 185 38 47 +185 38 47 140 40 59 105 35 55 76 28 47 58 20 33 102 29 42 +140 36 52 171 39 51 159 38 55 119 40 62 105 35 55 143 45 64 +166 48 64 159 46 62 159 46 62 144 40 57 100 28 46 89 29 45 +86 29 44 73 26 42 76 28 47 88 29 45 89 29 45 105 35 55 +105 35 55 88 29 45 67 27 47 61 26 49 56 25 45 52 25 42 +52 25 42 52 25 42 49 25 40 30 10 18 7 2 4 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 1 +97 28 42 185 38 47 142 31 41 69 20 33 57 25 43 50 18 33 +42 17 30 10 4 6 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 31 10 18 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 139 38 56 79 27 44 71 23 37 124 30 42 185 38 47 +185 38 47 185 38 47 185 38 47 166 48 64 152 49 69 166 48 64 +185 38 47 166 48 64 143 45 64 112 32 46 117 31 42 141 31 41 +144 31 43 123 33 48 79 27 44 60 26 44 76 28 47 76 28 47 +105 35 55 105 35 55 89 29 45 88 29 45 73 26 42 60 26 44 +56 25 45 52 25 42 49 25 40 49 25 40 40 15 29 16 5 9 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 35 12 21 +132 30 39 185 38 47 132 30 39 71 23 37 50 18 33 49 25 40 +42 17 30 30 10 18 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 36 11 20 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 110 32 46 71 23 37 142 31 41 185 38 47 185 38 47 +185 38 47 171 39 51 166 48 64 166 48 64 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 140 36 52 86 29 44 59 25 44 67 27 47 +89 29 45 113 41 62 113 41 62 97 28 42 100 28 46 88 29 45 +65 27 44 58 26 51 56 25 45 49 25 40 52 25 42 42 17 30 +28 8 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 41 14 25 97 28 42 +148 30 39 162 38 49 117 31 42 67 23 37 49 25 40 42 17 30 +42 17 30 35 12 21 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 41 13 21 185 38 47 185 38 47 185 38 47 185 38 47 +159 35 49 71 23 37 141 31 41 185 38 47 185 38 47 141 31 41 +117 31 42 113 41 62 118 50 79 113 41 62 166 48 64 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 129 39 59 67 27 47 56 25 45 +76 28 47 105 35 55 133 35 50 112 32 46 110 32 46 100 28 46 +89 29 45 86 29 44 73 26 48 62 26 49 58 26 51 52 25 42 +49 25 40 39 15 26 9 3 6 0 0 0 0 0 0 3 1 1 +10 4 6 20 6 10 47 16 28 69 20 33 135 30 41 185 38 47 +185 38 47 152 32 42 97 28 42 63 22 36 50 18 33 48 19 31 +50 18 33 39 15 26 4 0 1 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 41 13 21 185 38 47 185 38 47 185 38 47 171 39 51 +97 28 42 132 31 43 185 38 47 171 39 51 102 29 42 118 50 79 +167 135 158 210 187 199 171 141 162 108 54 86 152 40 56 185 38 47 +185 38 47 185 38 47 171 39 51 121 30 42 132 30 39 152 32 42 +185 38 47 185 38 47 185 38 47 185 38 47 118 34 52 61 26 49 +67 27 47 89 29 45 128 33 49 128 33 49 123 33 48 120 32 46 +120 32 46 120 32 46 100 28 46 76 28 47 73 26 48 65 27 44 +68 23 35 66 24 37 65 27 44 47 16 28 28 8 14 69 20 33 +102 29 42 132 30 39 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 124 30 42 79 27 44 60 22 36 51 24 39 50 18 33 +48 19 31 40 15 29 9 3 6 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 36 11 20 185 38 47 185 38 47 185 38 47 102 29 42 +117 31 42 185 38 47 171 39 51 112 32 46 148 114 145 233 215 221 +253 249 249 238 222 225 148 114 145 113 41 62 185 38 47 185 38 47 +171 39 51 125 35 52 134 49 77 148 114 145 148 114 145 131 80 105 +113 41 62 154 38 52 185 38 47 185 38 47 171 39 51 88 29 45 +76 28 47 105 35 55 119 34 51 133 35 50 135 32 45 128 33 49 +128 33 49 133 35 50 118 34 52 100 28 46 89 29 45 86 29 44 +100 28 46 121 30 42 135 30 41 135 30 41 148 30 39 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +132 30 39 73 26 42 65 27 44 65 27 44 57 25 43 49 25 40 +42 17 30 50 18 33 18 7 11 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 7 1 3 141 31 41 185 38 47 145 31 41 97 28 42 +185 38 47 162 38 49 110 32 46 191 162 183 248 240 240 253 249 249 +243 232 234 210 187 199 131 80 105 119 40 62 185 38 47 185 38 47 +128 41 59 159 124 153 226 205 215 231 213 218 226 205 215 228 209 216 +171 141 162 118 50 79 138 32 45 185 38 47 185 38 47 119 34 51 +88 29 45 119 40 62 119 40 62 125 41 61 124 34 50 128 33 49 +128 33 49 128 33 49 124 34 50 118 34 52 110 32 46 119 34 51 +136 33 46 142 31 41 152 32 42 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 171 39 51 132 30 39 +97 28 42 71 23 37 66 25 40 57 25 43 49 25 40 42 17 30 +42 17 30 50 18 33 18 7 11 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 57 17 29 171 39 51 97 28 42 148 30 39 +185 38 47 97 28 42 167 135 158 253 249 249 253 249 249 243 232 234 +241 229 231 202 176 194 113 41 62 171 39 51 185 38 47 133 35 50 +167 135 158 253 249 249 245 236 237 234 219 224 231 213 218 228 209 216 +219 198 209 148 114 145 86 29 44 171 39 51 185 38 47 159 38 55 +119 34 51 122 42 63 132 43 63 122 42 63 121 37 56 128 33 49 +123 33 48 119 34 51 123 32 47 110 32 46 118 34 52 140 36 52 +145 31 41 152 32 42 162 38 49 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 152 32 42 132 30 39 97 28 42 +71 23 37 71 23 37 65 27 44 56 25 45 49 25 40 42 17 30 +42 17 30 42 17 30 18 6 10 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 22 7 12 113 41 62 102 29 42 185 38 47 +116 30 44 148 114 145 245 236 237 253 249 249 248 240 240 238 224 228 +243 232 234 167 135 158 141 31 41 185 38 47 162 38 49 141 92 121 +245 236 237 253 249 249 248 240 240 243 232 234 238 224 228 233 215 221 +226 205 215 171 141 162 108 54 86 135 30 41 185 38 47 185 38 47 +152 40 56 135 40 58 141 42 59 135 40 58 128 33 49 133 35 50 +133 35 50 128 33 49 123 32 47 105 35 55 105 35 55 131 38 56 +138 32 45 136 31 43 148 30 39 152 32 42 185 38 47 185 38 47 +185 38 47 171 39 51 142 31 41 117 31 42 97 28 42 86 29 44 +68 23 35 66 24 37 58 24 38 49 25 40 48 19 31 42 17 30 +49 25 40 42 17 30 10 4 6 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 7 1 3 73 26 42 145 39 54 144 31 43 +120 71 102 238 222 225 253 249 249 253 249 249 241 229 231 241 229 231 +228 209 216 131 80 105 171 39 51 185 38 47 131 80 105 238 224 228 +253 249 249 248 240 240 238 222 225 238 222 225 238 224 228 238 222 225 +228 209 216 180 150 172 108 66 98 102 29 42 185 38 47 185 38 47 +185 38 47 159 38 55 171 39 51 150 36 50 133 35 50 133 31 44 +129 33 45 120 32 46 128 33 49 128 33 49 121 37 56 128 33 49 +128 33 49 125 31 44 141 31 41 148 30 39 152 32 42 148 30 39 +135 30 41 132 30 39 124 30 42 102 29 42 86 29 44 71 23 37 +65 27 44 56 25 45 51 24 39 49 25 40 50 18 33 48 19 31 +42 17 30 39 15 26 7 2 4 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 97 28 42 162 38 49 126 41 60 +210 187 199 253 249 249 253 249 249 248 240 240 238 224 228 245 236 237 +196 169 185 136 33 46 185 38 47 143 45 64 196 169 185 253 249 249 +253 249 249 243 232 234 238 224 228 238 224 228 238 224 228 238 222 225 +231 213 218 180 150 172 138 88 121 89 29 45 171 39 51 185 38 47 +185 38 47 185 38 47 171 39 51 150 36 50 142 33 47 135 31 44 +125 31 44 117 31 44 112 32 46 138 34 49 117 31 44 120 32 46 +120 32 46 117 31 44 136 31 43 144 31 43 148 30 39 121 30 42 +117 31 42 102 29 42 97 28 42 73 26 42 65 27 44 60 26 44 +52 25 42 49 25 40 50 18 33 50 18 33 49 25 40 42 17 30 +38 12 21 28 8 14 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 18 7 11 117 31 42 132 30 39 167 135 158 +248 240 240 253 249 249 253 249 249 245 236 237 245 236 237 245 236 237 +148 114 145 159 38 55 159 38 55 171 141 162 253 249 249 253 249 249 +248 240 240 243 232 234 233 215 221 234 219 224 241 229 231 231 213 218 +238 222 225 191 162 183 137 96 131 105 35 55 162 38 49 185 38 47 +185 38 47 171 39 51 171 39 51 159 35 49 142 33 47 133 32 44 +123 33 45 117 31 42 132 30 39 123 32 47 118 34 52 118 34 52 +112 32 46 117 31 44 117 31 44 117 31 42 117 31 42 97 28 42 +97 28 42 86 29 44 71 23 37 60 25 40 52 25 42 56 25 45 +49 25 40 49 25 40 49 25 40 49 25 40 48 19 31 42 17 30 +47 16 28 7 2 4 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 41 13 21 140 36 52 110 32 46 226 205 215 +233 215 221 233 215 221 241 229 231 241 229 231 243 232 234 233 215 221 +152 49 69 159 35 49 131 80 105 241 229 231 253 249 249 248 240 240 +241 229 231 241 229 231 238 222 225 238 224 228 241 229 231 238 222 225 +233 215 221 202 176 194 148 114 145 108 54 86 152 32 42 185 38 47 +185 38 47 185 38 47 171 39 51 153 36 50 139 38 56 128 33 49 +117 31 44 125 31 44 131 32 43 117 31 44 113 41 62 110 32 46 +117 31 44 133 32 44 117 31 42 89 29 45 89 29 45 86 29 44 +71 23 37 64 25 43 60 26 44 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 49 25 40 50 18 33 42 17 30 35 12 21 +16 5 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 57 17 29 135 30 41 159 124 153 196 169 185 +191 162 183 140 102 127 108 66 98 210 187 199 248 240 240 196 169 185 +143 45 64 159 46 63 180 150 172 245 236 237 253 249 249 248 240 240 +241 229 231 231 213 218 241 229 231 241 229 231 238 222 225 238 222 225 +228 209 216 191 162 183 159 124 153 108 54 86 141 31 41 185 38 47 +185 38 47 185 38 47 185 38 47 150 36 50 140 36 52 125 31 44 +123 33 45 121 30 42 116 30 44 102 29 42 110 32 46 112 32 46 +110 32 46 102 29 42 86 29 44 73 26 42 67 27 47 56 25 45 +52 25 42 52 25 42 52 25 42 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 42 17 30 40 15 29 35 12 21 10 4 6 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 97 28 42 152 49 69 191 162 183 141 92 121 +228 209 216 108 66 98 50 18 33 140 102 127 243 232 234 131 80 105 +148 37 53 134 49 77 180 150 172 191 162 183 196 169 185 210 187 199 +241 229 231 233 215 221 231 213 218 238 224 228 238 222 225 238 222 225 +233 215 221 202 176 194 159 124 153 93 57 98 124 30 42 185 38 47 +185 38 47 185 38 47 185 38 47 146 36 50 133 35 50 124 30 42 +125 31 44 124 30 42 110 32 46 102 29 42 97 28 42 97 28 42 +97 28 42 71 23 37 57 25 43 56 25 45 56 25 45 52 25 42 +49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 49 25 40 31 10 18 7 2 4 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 4 0 1 102 29 42 141 92 121 131 80 105 180 150 172 +196 169 185 58 26 51 52 25 42 120 71 102 202 176 194 121 37 56 +135 40 58 71 23 37 108 54 86 210 187 199 108 54 86 89 29 45 +202 176 194 243 232 234 238 224 228 234 219 224 241 229 231 238 222 225 +233 215 221 202 176 194 159 124 153 108 54 86 124 30 42 185 38 47 +185 38 47 185 38 47 185 38 47 162 38 49 159 35 49 135 31 44 +125 31 44 121 30 42 117 31 42 97 28 42 86 29 44 67 23 37 +52 24 38 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 35 12 21 7 2 4 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 10 4 6 97 28 42 131 80 105 76 28 47 137 96 131 +93 57 98 54 21 35 50 18 33 108 54 86 159 124 153 125 35 52 +89 29 45 48 19 31 159 124 153 191 162 183 51 24 39 54 19 32 +131 80 105 243 232 234 245 236 237 243 232 234 241 229 231 238 224 228 +228 209 216 202 176 194 167 135 158 93 57 98 121 30 42 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 152 32 42 129 33 45 +121 30 42 117 31 42 97 28 42 71 23 37 67 22 35 54 21 35 +52 24 38 56 25 45 52 25 42 49 25 40 52 25 42 49 25 40 +49 25 40 49 25 40 49 25 40 48 19 31 50 18 33 42 17 30 +50 18 33 22 7 12 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 10 4 6 86 29 44 79 27 44 60 25 40 49 25 40 +57 25 43 56 25 45 52 20 33 108 54 86 134 49 77 129 39 59 +71 23 37 60 25 40 196 169 185 167 135 158 50 18 33 52 25 42 +76 28 47 219 198 209 248 240 240 241 229 231 245 236 237 238 224 228 +231 213 218 202 176 194 159 124 153 93 57 98 132 30 39 185 38 47 +185 38 47 185 38 47 185 38 47 171 39 51 152 32 42 132 30 39 +124 30 42 102 29 42 86 29 44 71 23 37 67 22 35 52 24 38 +52 25 42 49 25 40 52 25 42 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 49 25 40 42 17 30 42 17 30 28 8 14 +22 7 12 3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 57 17 29 138 37 53 65 27 44 61 26 49 56 25 45 +51 24 39 49 25 40 58 24 38 88 29 45 144 40 57 135 40 58 +63 22 36 52 25 42 167 135 158 120 71 102 49 25 40 56 25 45 +67 27 47 196 169 185 248 240 240 243 232 234 245 236 237 238 224 228 +219 198 209 196 169 185 137 96 131 76 28 47 159 35 49 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 152 32 42 142 31 41 +135 30 41 121 30 42 97 28 42 71 23 37 61 21 34 50 18 33 +52 25 42 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 48 19 31 35 12 21 16 5 9 1 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +22 7 12 152 32 42 148 37 50 66 25 40 67 27 47 61 26 49 +52 25 42 52 25 42 54 21 35 79 27 44 154 39 54 113 41 62 +60 25 40 61 26 49 58 26 51 62 26 49 56 25 45 56 25 45 +61 21 34 167 135 158 248 240 240 238 222 225 243 232 234 238 222 225 +228 209 216 180 150 172 137 96 131 79 28 45 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 171 39 51 159 35 49 135 30 41 +132 30 39 124 30 42 102 29 42 71 23 37 67 22 35 48 16 29 +49 25 40 52 25 42 49 25 40 49 25 40 49 25 40 50 18 33 +42 17 30 41 14 25 22 7 12 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 57 17 29 +162 38 49 185 38 47 171 39 51 97 28 42 60 23 38 65 27 44 +60 26 44 60 26 44 57 17 29 131 32 43 159 38 55 89 29 45 +57 25 43 61 26 49 56 25 45 61 26 49 61 26 49 61 26 49 +65 27 44 171 141 162 243 232 234 243 232 234 245 236 237 233 215 221 +219 198 209 171 141 162 120 71 102 100 28 46 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 171 39 51 150 36 50 135 31 44 +121 30 42 121 30 42 102 29 42 86 29 44 61 21 34 56 22 36 +49 25 40 52 25 42 52 25 42 49 25 40 49 25 40 42 17 30 +48 19 31 22 7 12 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 10 4 6 97 28 42 185 38 47 +185 38 47 185 38 47 185 38 47 159 35 49 117 31 42 71 23 37 +71 23 37 71 23 37 124 30 42 185 38 47 171 39 51 97 28 42 +60 26 44 62 26 49 52 25 42 61 26 49 61 26 49 67 27 47 +73 26 48 171 141 162 243 232 234 245 236 237 243 232 234 231 213 218 +202 176 194 159 124 153 93 57 98 131 38 56 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 171 39 51 159 35 49 135 32 45 +124 30 42 117 31 42 102 29 42 71 23 37 61 21 34 52 24 38 +49 25 40 49 25 40 52 25 42 52 25 42 49 25 40 40 15 29 +36 11 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 7 2 4 102 29 42 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 133 32 44 71 23 37 +73 26 42 110 32 46 153 36 50 171 39 51 171 39 51 112 32 46 +67 27 47 61 26 49 61 26 49 61 26 49 56 25 45 57 25 43 +105 35 55 210 187 199 245 236 237 241 229 231 231 213 218 233 215 221 +210 187 199 146 108 143 76 28 47 159 35 49 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 152 32 42 138 32 45 +117 31 42 110 32 46 97 28 42 71 23 37 60 23 38 51 24 39 +51 24 39 49 25 40 52 25 42 52 25 42 49 25 40 49 25 40 +16 5 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 38 12 21 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 135 30 41 124 30 42 152 32 42 +171 39 51 171 39 51 153 36 50 136 31 43 171 39 51 171 39 51 +89 29 45 67 27 47 56 25 45 60 26 44 60 22 36 67 22 35 +159 124 153 243 232 234 243 232 234 233 215 221 231 213 218 226 205 215 +180 150 172 108 66 98 97 28 42 185 38 47 185 38 47 185 38 47 +185 38 47 171 39 51 171 39 51 171 39 51 153 36 50 135 32 45 +123 32 47 102 29 42 97 28 42 71 23 37 60 25 40 57 25 43 +49 25 40 49 25 40 52 25 42 52 25 42 49 25 40 49 25 40 +16 5 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 10 4 6 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 7 2 4 110 32 46 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 148 30 39 148 30 39 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 171 39 51 162 38 49 185 38 47 +120 32 46 71 23 37 56 25 45 57 25 43 61 21 34 108 54 86 +219 198 209 241 229 231 233 215 221 238 222 225 226 205 215 191 162 183 +137 96 131 105 35 55 171 39 51 185 38 47 185 38 47 185 38 47 +185 38 47 159 38 55 159 38 55 162 38 49 144 40 57 136 33 46 +112 32 46 110 32 46 86 29 44 73 26 42 64 24 39 57 25 43 +56 25 45 58 26 51 58 26 51 58 26 51 49 25 40 49 25 40 +9 3 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 3 1 1 108 66 98 120 71 102 140 102 127 +93 57 98 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 10 4 6 153 36 50 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 135 30 41 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 159 38 55 159 38 55 +150 39 55 105 35 55 66 25 40 60 23 38 76 28 47 167 135 158 +248 240 240 253 249 249 243 232 234 233 215 221 202 176 194 148 114 145 +108 66 98 121 37 56 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 171 39 51 171 39 51 171 39 51 150 39 55 138 34 49 +112 32 46 102 29 42 97 28 42 71 23 37 67 23 37 60 26 44 +61 26 49 58 26 51 58 26 51 52 25 42 49 25 40 49 25 40 +9 3 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 1 0 0 93 57 98 202 176 194 +167 135 158 108 66 98 42 17 30 0 0 0 0 0 0 0 0 0 +0 0 0 7 2 4 152 32 42 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 132 30 39 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 152 40 56 128 41 59 +121 37 56 119 40 62 105 35 55 79 27 44 105 35 55 148 114 145 +180 150 172 191 162 183 148 114 145 141 92 121 108 54 86 108 54 86 +86 29 44 124 37 57 159 35 49 171 39 51 185 38 47 185 38 47 +153 36 50 159 46 62 171 39 51 171 39 51 152 32 42 135 32 45 +110 32 46 102 29 42 86 29 44 71 23 37 52 24 38 60 26 44 +58 26 51 58 26 51 58 26 51 52 25 42 49 25 40 49 25 40 +9 3 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 138 88 121 +146 108 143 58 26 51 58 26 51 10 4 6 0 0 0 0 0 0 +0 0 0 7 2 4 141 31 41 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 135 30 41 148 30 39 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 171 39 51 143 45 64 123 43 68 +113 41 62 105 35 55 105 35 55 136 44 65 131 38 56 105 35 55 +105 35 55 116 30 44 116 30 44 102 29 42 117 31 42 124 30 42 +132 30 39 153 36 50 159 35 49 171 39 51 171 39 51 159 35 49 +153 36 50 144 40 57 150 39 55 154 39 54 159 38 55 123 33 45 +102 29 42 97 28 42 86 29 44 73 26 42 51 24 39 58 26 51 +58 26 51 58 26 51 52 25 42 52 25 42 49 25 40 39 15 26 +4 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 58 26 51 +196 169 185 93 57 98 58 26 51 42 17 30 3 1 1 0 0 0 +0 0 0 7 2 4 97 28 42 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 116 30 44 132 30 39 185 38 47 +185 38 47 171 39 51 159 46 63 143 45 66 134 49 77 108 54 86 +108 54 86 108 54 86 76 28 47 119 34 51 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +171 39 51 154 39 54 144 40 57 140 36 52 142 33 47 120 32 46 +102 29 42 89 29 45 86 29 44 65 27 44 56 25 45 58 26 51 +58 26 51 58 26 51 52 25 42 52 25 42 52 25 42 35 12 21 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 25 40 +171 141 162 120 71 102 93 57 98 93 57 98 18 7 11 0 0 0 +0 0 0 0 0 0 36 11 20 171 39 51 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 159 38 55 112 32 46 110 32 46 +117 31 44 119 40 62 113 41 62 89 29 45 88 29 45 76 28 47 +76 28 47 105 35 55 76 28 47 110 32 46 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 171 39 51 159 35 49 159 35 49 136 33 46 110 32 46 +102 29 42 86 29 44 73 26 42 59 25 44 61 26 49 58 26 51 +58 26 51 58 26 51 58 26 51 49 25 40 49 25 40 30 10 18 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +49 25 40 22 7 12 7 2 4 93 57 98 93 57 98 0 0 0 +0 0 0 0 0 0 18 7 11 133 35 50 185 38 47 185 38 47 +185 38 47 154 39 54 141 40 57 159 46 63 166 48 64 138 34 49 +116 30 44 105 35 55 113 41 62 105 35 55 105 35 55 121 37 56 +105 35 55 113 41 62 113 41 62 135 40 58 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 154 38 52 128 33 49 101 29 42 +86 29 44 79 28 45 67 27 47 61 26 49 56 25 45 52 25 42 +58 26 51 58 26 51 52 25 42 49 25 40 49 25 40 16 5 9 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 25 40 +138 88 121 148 114 145 140 102 127 148 114 145 30 10 18 0 0 0 +0 0 0 0 0 0 0 0 0 18 7 11 137 96 131 40 15 29 +0 0 0 0 0 0 0 0 0 58 24 38 159 46 62 185 38 47 +185 38 47 141 43 61 89 29 45 86 29 44 140 36 52 185 38 47 +171 39 51 171 39 51 159 38 55 159 38 55 166 48 64 166 48 64 +166 48 64 159 46 63 152 49 69 166 48 64 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 159 35 49 124 34 50 118 34 52 100 28 46 +86 29 44 67 27 47 67 27 47 58 26 51 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 52 25 42 40 15 29 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39 15 26 +171 141 162 146 108 143 146 108 143 120 71 102 30 10 18 0 0 0 +0 0 0 0 0 0 0 0 0 1 0 0 108 66 98 95 59 101 +0 0 0 0 0 0 0 0 0 0 0 0 35 12 21 117 31 44 +171 39 51 171 39 51 135 40 58 79 28 45 77 27 45 141 40 57 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 171 39 51 +171 39 51 159 38 55 133 35 50 121 37 56 105 35 55 105 35 55 +88 29 45 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +58 26 51 58 26 51 49 25 40 39 15 26 7 2 4 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 2 4 +140 102 127 76 28 47 58 26 51 93 57 98 39 15 26 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 30 10 18 137 96 131 +39 15 26 0 0 0 0 0 0 0 0 0 0 0 0 4 0 1 +79 27 44 144 45 67 159 46 63 134 49 77 88 29 45 89 29 45 +152 40 56 171 39 51 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 166 48 64 159 46 63 159 38 55 159 38 55 150 39 55 +140 40 59 119 40 62 113 41 62 105 35 55 105 35 55 88 29 45 +67 27 47 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +58 26 51 52 25 42 40 15 29 7 2 4 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +140 102 127 93 57 98 76 28 47 58 26 51 93 57 98 30 10 18 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 148 114 145 +58 26 51 3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 28 8 14 61 21 34 105 35 55 118 50 79 88 29 45 +102 29 42 152 40 56 166 48 64 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 166 48 64 159 46 63 +141 43 61 125 41 61 105 35 55 105 35 55 105 35 55 105 35 55 +105 35 55 122 42 63 113 41 62 105 35 55 88 29 45 73 26 48 +67 27 47 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +52 25 42 35 12 21 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +140 102 127 49 25 40 35 12 21 58 26 51 93 57 98 58 26 51 +49 25 40 30 10 18 16 5 9 0 0 0 52 25 42 138 88 121 +40 15 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 16 5 9 51 24 39 105 35 55 +76 28 47 105 35 55 143 45 64 166 48 64 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 171 39 51 166 48 64 159 46 63 134 49 77 118 50 79 +113 41 62 66 24 37 61 21 34 79 27 44 76 28 47 76 28 47 +88 29 45 105 35 55 105 35 55 76 28 47 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +40 15 29 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 42 17 30 93 57 98 +58 26 51 58 26 51 58 26 51 93 57 98 108 66 98 58 26 51 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 5 9 +42 17 30 77 27 45 105 35 55 128 41 59 166 48 64 166 48 64 +166 48 64 185 38 47 171 39 51 171 39 51 166 48 64 159 46 62 +159 38 55 159 46 62 139 44 66 118 50 79 108 54 86 76 28 47 +73 26 42 73 26 42 89 29 45 113 41 62 118 50 79 123 42 67 +122 42 63 113 41 62 105 35 55 73 26 48 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 39 15 26 +3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 10 4 6 140 102 127 159 124 153 120 71 102 +140 102 127 140 102 127 131 80 105 131 80 105 108 66 98 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39 15 26 +58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 42 17 30 +1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 18 7 11 47 16 28 79 27 44 119 40 62 145 47 69 +152 49 69 152 49 69 152 49 69 145 47 69 145 47 69 136 44 65 +125 41 61 113 41 62 105 35 55 76 28 47 76 28 47 97 28 42 +117 31 44 128 41 59 128 42 60 128 42 60 124 42 65 113 41 62 +105 35 55 105 35 55 105 35 55 88 29 45 76 28 47 58 26 51 +58 26 51 58 26 51 58 26 51 49 25 40 30 10 18 4 0 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 39 15 26 202 176 194 226 205 215 +202 176 194 191 162 183 159 124 153 131 80 105 35 12 21 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +52 25 42 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +22 7 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 9 3 6 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 18 6 10 73 26 42 105 35 55 +105 35 55 113 41 62 113 41 62 113 41 62 113 41 62 105 35 55 +105 35 55 76 28 47 79 27 44 101 29 42 129 33 45 150 39 55 +144 40 57 136 33 46 125 35 52 113 41 62 105 35 55 105 35 55 +105 35 55 105 35 55 88 29 45 76 28 47 58 26 51 58 26 51 +58 26 51 52 25 42 49 25 40 22 7 12 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 18 7 11 159 124 153 +146 108 143 146 108 143 148 114 145 108 66 98 10 4 6 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 2 4 +93 57 98 93 63 110 58 26 51 58 26 51 58 26 51 58 26 51 +56 25 45 18 7 11 7 2 4 3 1 1 39 15 26 124 30 42 +152 32 42 159 38 55 171 39 51 97 28 42 7 1 3 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 4 0 1 57 17 29 +121 37 56 105 35 55 100 28 46 97 28 42 86 29 44 86 29 44 +97 28 42 117 31 42 141 33 45 153 36 50 153 36 50 140 36 52 +128 33 49 105 35 55 105 35 55 105 35 55 89 29 45 105 35 55 +105 35 55 76 28 47 67 27 47 58 26 51 58 26 51 58 26 51 +52 25 42 49 25 40 16 5 9 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 2 4 +93 57 98 93 63 110 93 57 98 108 66 98 131 80 105 42 17 30 +49 25 40 76 28 47 67 27 47 42 17 30 76 28 47 140 102 127 +171 141 162 93 63 110 58 26 51 42 17 30 58 26 51 58 26 51 +58 26 51 58 26 51 76 28 47 65 27 44 152 32 42 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 112 32 46 39 15 26 +3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +20 6 10 69 20 33 139 37 53 159 38 55 159 35 49 162 38 49 +159 38 55 162 38 49 150 36 50 142 32 45 124 34 50 105 35 55 +105 35 55 88 29 45 105 35 55 105 35 55 89 29 45 89 29 45 +105 35 55 88 29 45 76 28 47 58 26 51 58 26 51 52 25 42 +52 25 42 39 15 26 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 10 4 6 58 26 51 58 26 51 58 26 51 93 57 98 +108 66 98 131 80 105 137 96 131 159 124 153 167 135 158 137 96 131 +58 26 51 22 7 12 7 2 4 0 0 0 9 3 6 35 12 21 +49 25 40 58 26 51 58 26 51 112 32 46 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 153 36 50 105 35 55 58 26 51 +18 7 11 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 3 1 1 42 17 30 97 28 42 119 34 51 128 33 49 +131 38 56 122 40 63 105 35 55 105 35 55 88 29 45 76 28 47 +76 28 47 73 26 42 105 35 55 105 35 55 105 35 55 105 35 55 +105 35 55 105 35 55 89 29 45 76 28 47 58 26 51 52 25 42 +49 25 40 18 7 11 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 3 1 1 22 7 12 18 7 11 22 7 12 +30 10 18 30 10 18 40 15 29 42 17 30 30 10 18 9 3 6 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +28 8 14 58 26 51 58 26 51 118 34 52 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 118 34 52 58 26 51 58 26 51 +49 25 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 3 1 1 57 17 29 79 27 44 71 23 37 +73 26 42 71 23 37 79 27 44 67 23 37 68 23 35 79 27 44 +86 29 44 105 35 55 124 35 54 128 33 49 135 32 45 138 32 45 +128 33 49 119 34 51 100 28 46 86 29 44 58 26 51 58 26 51 +48 19 31 16 5 9 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 39 15 26 58 26 51 76 28 47 152 32 42 185 38 47 +185 38 47 185 38 47 135 40 58 76 28 47 58 26 51 58 26 51 +30 10 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 57 17 29 139 44 68 139 38 56 +135 30 41 117 31 42 116 30 44 123 32 47 125 35 52 135 40 58 +135 40 58 140 36 52 152 32 42 162 38 49 162 38 49 185 38 47 +162 38 49 152 32 42 128 33 49 100 28 46 62 26 49 58 26 51 +52 25 42 22 7 12 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 30 10 18 76 28 47 119 40 62 102 29 42 142 34 47 +150 36 50 128 41 59 105 35 55 76 28 47 58 26 51 48 19 31 +1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 7 2 4 118 34 52 159 46 63 171 39 51 +185 38 47 171 39 51 171 39 51 171 39 51 171 39 51 171 39 51 +171 39 51 162 38 49 171 39 51 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 118 34 52 76 28 47 56 25 45 +58 26 51 30 10 18 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 31 10 18 154 39 54 185 38 47 150 36 50 123 33 45 +112 32 46 105 35 55 105 35 55 76 28 47 61 26 49 67 27 47 +1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 1 0 0 61 21 34 166 48 64 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 138 32 45 76 28 47 56 25 45 +49 25 40 42 17 30 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 117 31 42 185 38 47 185 38 47 185 38 47 185 38 47 +162 38 49 125 41 61 105 35 55 67 27 47 61 21 34 61 26 49 +39 15 26 22 7 12 18 7 11 7 2 4 3 1 1 3 1 1 +30 10 18 66 25 40 139 38 56 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 100 28 46 61 26 49 +49 25 40 49 25 40 20 6 10 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +28 8 14 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 139 44 68 108 54 86 73 26 48 57 25 43 58 26 51 +58 26 51 58 26 51 58 26 51 56 25 45 52 25 42 56 25 45 +73 26 48 113 41 62 152 49 69 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 162 38 49 162 38 49 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 132 30 39 76 28 47 +58 26 51 49 25 40 35 12 21 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +41 13 21 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +166 48 64 134 49 77 108 54 86 76 28 47 67 27 47 73 26 48 +58 26 51 58 26 51 58 26 51 58 26 51 61 26 49 61 26 49 +108 54 86 105 35 55 105 35 55 119 34 51 133 32 44 148 30 39 +152 32 42 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 159 35 49 159 35 49 159 35 49 159 35 49 171 39 51 +185 38 47 185 38 47 185 38 47 171 39 51 125 31 44 79 27 44 +58 26 51 49 25 40 40 15 29 10 4 6 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +36 11 20 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +152 40 56 118 50 79 108 54 86 76 28 47 66 25 40 76 28 47 +58 26 51 58 26 51 58 26 51 58 26 51 61 26 49 76 28 47 +113 41 62 133 35 50 150 39 55 162 38 49 162 38 49 148 30 39 +132 30 39 132 30 39 132 30 39 154 39 54 159 46 62 154 39 54 +146 36 50 138 34 49 138 37 53 135 40 58 141 40 57 150 36 50 +159 35 49 171 39 51 185 38 47 150 36 50 105 35 55 79 28 45 +58 26 51 52 25 42 49 25 40 30 10 18 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +7 2 4 117 31 44 171 39 51 185 38 47 166 48 64 145 47 69 +118 50 79 108 54 86 105 35 55 73 26 42 76 28 47 105 35 55 +76 28 47 58 26 51 58 26 51 58 26 51 52 24 38 64 24 39 +121 37 56 171 39 51 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 152 32 42 121 30 42 102 29 42 124 34 50 +124 35 54 113 41 62 113 41 62 113 41 62 113 41 62 118 34 52 +133 35 50 159 35 49 159 35 49 133 35 50 88 29 45 79 28 45 +58 26 51 52 25 42 49 25 40 42 17 30 10 4 6 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 16 5 9 105 35 55 113 41 62 100 28 46 89 29 45 +76 28 47 76 28 47 73 26 42 76 28 47 105 35 55 105 35 55 +58 26 51 58 26 51 58 26 51 58 26 51 56 25 45 60 22 36 +101 29 42 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 124 30 42 86 29 44 +105 35 55 105 35 55 105 35 55 76 28 47 76 28 47 105 35 55 +100 28 46 116 30 44 152 32 42 138 34 49 79 28 45 62 26 49 +52 25 42 52 25 42 49 25 40 49 25 40 28 8 14 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 7 2 4 60 26 44 110 32 46 102 29 42 +97 28 42 118 34 52 121 37 56 113 41 62 113 41 62 73 26 48 +58 26 51 58 26 51 58 26 51 61 26 49 62 26 49 52 25 42 +71 23 37 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 171 39 51 133 35 50 +97 28 42 86 29 44 76 28 47 73 26 48 76 28 47 88 29 45 +69 20 33 79 27 44 139 38 56 125 35 52 76 28 47 58 26 51 +58 26 51 52 25 42 49 25 40 49 25 40 30 10 18 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 1 0 0 57 17 29 152 32 42 +185 38 47 166 48 64 123 43 68 108 54 86 105 35 55 58 26 51 +58 26 51 58 24 38 58 26 51 58 26 51 56 25 45 54 19 32 +105 35 55 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +159 35 49 117 31 42 86 29 44 71 23 37 71 23 37 67 23 37 +71 23 37 116 30 44 138 37 53 105 35 55 76 28 47 58 26 51 +58 26 51 52 25 42 49 25 40 49 25 40 35 12 21 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 28 8 14 +102 29 42 110 32 46 105 35 55 76 28 47 65 27 44 61 26 49 +49 25 40 56 25 45 58 26 51 58 26 51 58 20 33 71 23 37 +159 38 55 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 171 39 51 150 36 50 128 33 49 118 34 52 89 29 45 +89 29 45 105 35 55 89 29 45 73 26 48 61 26 49 58 26 51 +52 25 42 52 25 42 49 25 40 49 25 40 39 15 26 1 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 3 1 1 10 4 6 35 12 21 58 26 51 +58 26 51 58 26 51 58 26 51 49 25 40 69 20 33 171 39 51 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 171 39 51 +171 39 51 151 38 54 133 35 50 121 37 56 105 35 55 105 35 55 +76 28 47 62 26 49 73 26 48 58 26 51 58 26 51 58 26 51 +52 25 42 52 25 42 49 25 40 49 25 40 40 15 29 9 3 6 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 4 6 +51 24 39 56 25 45 66 25 40 110 32 46 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 171 39 51 139 37 53 +125 35 52 118 34 52 105 35 55 105 35 55 88 29 45 76 28 47 +62 26 49 61 26 49 62 26 49 58 26 51 56 25 45 49 25 40 +52 25 42 52 25 42 49 25 40 42 17 30 42 17 30 22 7 12 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +10 4 6 71 23 37 119 34 51 159 38 55 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 152 32 42 138 34 46 118 34 52 +105 35 55 89 29 45 76 28 47 76 28 47 73 26 48 73 26 48 +62 26 49 58 26 51 61 26 49 52 25 42 52 25 42 58 26 51 +58 26 51 49 25 40 49 25 40 48 19 31 50 18 33 28 8 14 +1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 48 16 29 125 41 61 128 42 60 145 39 54 171 39 51 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 159 38 55 141 40 57 124 34 50 100 28 46 89 29 45 +86 29 44 76 28 47 73 26 48 67 27 47 58 26 51 62 26 49 +58 26 51 56 25 45 56 25 45 58 26 51 58 26 51 58 26 51 +52 25 42 49 25 40 49 25 40 42 17 30 35 12 21 16 5 9 +1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 22 7 12 89 29 45 113 41 62 118 50 79 122 42 63 +132 43 63 144 40 57 159 38 55 153 36 50 171 39 51 150 39 55 +135 40 58 128 41 59 113 41 62 105 35 55 105 35 55 76 28 47 +73 26 48 73 26 48 73 26 48 58 26 51 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 52 25 42 49 25 40 +49 25 40 49 25 40 49 25 40 50 18 33 20 6 10 1 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 7 2 4 56 22 36 76 28 47 105 35 55 113 41 62 +118 50 79 113 41 62 123 43 68 113 41 62 122 40 63 119 40 62 +113 41 62 105 35 55 105 35 55 89 29 45 76 28 47 73 26 48 +73 26 48 73 26 48 58 26 51 58 26 51 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 52 25 42 52 25 42 49 25 40 +49 25 40 48 19 31 42 17 30 42 17 30 22 7 12 1 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 31 10 18 69 20 33 86 29 44 89 29 45 +105 35 55 105 35 55 118 50 79 113 41 62 113 41 62 105 35 55 +105 35 55 105 35 55 105 35 55 76 28 47 73 26 48 67 27 47 +58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +58 26 51 58 26 51 52 25 42 49 25 40 49 25 40 49 25 40 +49 25 40 50 18 33 40 15 29 48 16 29 16 5 9 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 86 29 44 97 28 42 105 35 55 88 29 45 +88 29 45 105 35 55 113 41 62 113 41 62 108 54 86 108 54 86 +105 35 55 105 35 55 76 28 47 76 28 47 73 26 48 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 52 25 42 +49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 50 18 33 +42 17 30 39 15 26 38 12 21 28 8 14 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 28 8 14 185 38 47 162 38 49 86 29 44 79 28 45 +76 28 47 113 41 62 118 50 79 108 54 86 108 54 86 108 54 86 +105 35 55 76 28 47 76 28 47 62 26 49 62 26 49 58 26 51 +58 26 51 56 25 45 52 25 42 52 25 42 49 25 40 49 25 40 +49 25 40 49 25 40 49 25 40 48 19 31 42 17 30 39 15 26 +39 15 26 38 12 21 42 17 30 28 8 14 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 57 17 29 185 38 47 185 38 47 171 39 51 119 34 51 +89 29 45 105 35 55 108 54 86 108 54 86 108 54 86 108 54 86 +73 26 48 73 26 48 62 26 49 73 26 48 58 26 51 58 26 51 +58 26 51 58 26 51 52 25 42 49 25 40 49 25 40 49 25 40 +49 25 40 48 19 31 42 17 30 39 15 26 39 15 26 42 17 30 +42 17 30 49 25 40 52 25 42 35 12 21 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 69 20 33 185 38 47 185 38 47 185 38 47 185 38 47 +154 39 54 119 40 62 105 35 55 105 35 55 105 35 55 105 35 55 +76 28 47 73 26 48 62 26 49 58 26 51 58 26 51 56 25 45 +61 26 49 49 25 40 42 17 30 50 18 33 48 19 31 42 17 30 +42 17 30 42 17 30 42 17 30 49 25 40 49 25 40 58 26 51 +58 26 51 49 25 40 48 19 31 42 17 30 7 1 3 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 69 20 33 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 166 48 64 135 40 58 105 35 55 88 29 45 88 29 45 +76 28 47 62 26 49 60 26 44 52 24 38 49 25 40 49 25 40 +57 25 43 58 24 38 52 24 38 49 25 40 49 25 40 50 18 33 +49 25 40 50 18 33 48 19 31 52 25 42 58 26 51 58 26 51 +58 26 51 52 25 42 48 19 31 42 17 30 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 41 13 21 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 152 40 56 119 34 51 105 35 55 +88 29 45 76 28 47 65 27 44 64 25 43 65 27 44 73 26 48 +76 28 47 76 28 47 76 28 47 60 26 44 48 19 31 42 17 30 +48 19 31 49 25 40 49 25 40 42 17 30 52 25 42 52 25 42 +52 25 42 52 25 42 49 25 40 48 19 31 10 4 6 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 16 5 9 152 32 42 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 166 48 64 134 49 77 +119 40 62 113 41 62 105 35 55 105 35 55 105 35 55 105 35 55 +105 35 55 88 29 45 105 35 55 76 28 47 73 26 48 58 24 38 +42 17 30 42 17 30 42 17 30 42 17 30 42 17 30 42 17 30 +52 25 42 56 25 45 49 25 40 49 25 40 30 10 18 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 9 3 6 102 29 42 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 171 39 51 +152 49 69 152 40 56 141 42 59 131 42 64 126 41 60 105 35 55 +105 35 55 113 41 62 105 35 55 88 29 45 105 35 55 105 35 55 +61 26 49 48 19 31 49 25 40 42 17 30 42 17 30 42 17 30 +48 19 31 49 25 40 49 25 40 49 25 40 40 15 29 3 1 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 16 5 9 159 35 49 185 38 47 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +166 48 64 166 48 64 159 46 63 144 40 57 136 44 65 125 41 61 +113 41 62 113 41 62 105 35 55 105 35 55 105 35 55 105 35 55 +76 28 47 61 26 49 48 19 31 48 19 31 48 19 31 42 17 30 +49 25 40 49 25 40 50 18 33 49 25 40 42 17 30 22 7 12 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 36 11 20 171 39 51 185 38 47 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 185 38 47 171 39 51 154 39 54 144 40 57 128 42 60 +113 41 62 105 35 55 105 35 55 105 35 55 105 35 55 76 28 47 +73 26 48 58 26 51 58 26 51 49 25 40 42 17 30 42 17 30 +42 17 30 49 25 40 42 17 30 48 19 31 49 25 40 42 17 30 +7 2 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 28 8 14 152 32 42 +185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 185 38 47 +185 38 47 171 39 51 159 46 62 144 40 57 144 40 57 129 39 59 +113 41 62 105 35 55 105 35 55 105 35 55 76 28 47 73 26 48 +58 26 51 58 26 51 58 26 51 58 26 51 52 25 42 42 17 30 +42 17 30 42 17 30 48 19 31 49 25 40 49 25 40 49 25 40 +35 12 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 36 11 20 +144 40 57 152 40 56 152 40 56 159 38 55 159 38 55 166 48 64 +166 48 64 159 46 63 145 47 69 130 43 63 119 40 62 105 35 55 +105 35 55 105 35 55 105 35 55 93 57 98 76 28 47 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 52 25 42 49 25 40 +49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +48 19 31 16 5 9 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 2 4 +86 29 44 118 50 79 118 50 79 118 50 79 129 39 59 136 44 65 +126 41 60 123 42 67 118 50 79 108 54 86 108 54 86 105 35 55 +105 35 55 105 35 55 76 28 47 73 26 48 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 49 25 40 49 25 40 52 25 42 +49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +48 19 31 36 11 20 9 3 6 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 10 18 +108 54 86 76 28 47 105 35 55 113 41 62 118 50 79 113 41 62 +113 41 62 108 54 86 105 35 55 105 35 55 105 35 55 105 35 55 +76 28 47 76 28 47 73 26 48 58 26 51 58 26 51 58 26 51 +58 26 51 58 26 51 52 25 42 52 25 42 52 25 42 49 25 40 +52 25 42 49 25 40 49 25 40 49 25 40 49 25 40 49 25 40 +42 17 30 50 18 33 35 12 21 4 0 1 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39 15 26 +105 35 55 105 35 55 113 41 62 105 35 55 76 28 47 76 28 47 +76 28 47 88 29 45 76 28 47 73 26 48 67 27 47 65 27 44 +62 26 49 62 26 49 67 27 47 60 26 44 52 25 42 52 25 42 +49 25 40 56 25 45 52 25 42 50 18 33 49 25 40 49 25 40 +52 25 42 56 25 45 52 25 42 49 25 40 49 25 40 49 25 40 +49 25 40 49 25 40 42 17 30 28 8 14 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 41 13 21 +129 39 59 152 40 56 159 46 63 133 43 64 105 35 55 66 25 40 +67 27 47 61 25 44 65 27 44 76 28 47 79 28 45 86 29 44 +89 29 45 88 29 45 67 27 47 49 25 40 52 24 38 52 24 38 +51 24 39 49 25 40 49 25 40 52 25 42 56 25 45 58 26 51 +62 26 49 56 25 45 42 17 30 35 12 21 42 17 30 49 25 40 +49 25 40 42 17 30 48 19 31 42 17 30 28 8 14 4 0 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 7 2 4 67 23 37 +134 49 77 152 49 69 166 48 64 166 48 64 145 39 54 102 29 42 +100 28 46 77 27 45 60 26 44 73 26 48 113 41 62 138 37 53 +133 35 50 131 38 56 105 35 55 76 28 47 66 25 40 76 28 47 +88 29 45 105 35 55 76 28 47 73 26 48 73 26 48 73 26 48 +73 26 48 48 19 31 10 4 6 0 0 0 18 7 11 49 25 40 +52 25 42 49 25 40 49 25 40 49 25 40 49 25 40 35 12 21 +10 4 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 42 17 30 76 28 47 +141 92 121 167 135 158 167 135 158 141 92 121 141 92 121 131 80 105 +116 30 44 105 35 55 67 23 37 76 28 47 118 50 79 152 49 69 +159 46 62 148 37 53 124 35 54 110 32 46 79 27 44 105 35 55 +136 44 65 134 49 77 118 50 79 108 54 86 76 28 47 58 26 51 +76 28 47 30 10 18 0 0 0 0 0 0 0 0 0 28 8 14 +52 25 42 52 25 42 52 25 42 52 25 42 52 25 42 49 25 40 +49 25 40 31 10 18 10 4 6 1 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 1 0 0 18 7 11 108 54 86 89 29 45 +133 43 64 142 34 47 137 33 45 128 41 59 134 49 77 141 92 121 +143 45 64 121 37 56 73 26 42 118 50 79 131 80 105 141 92 121 +141 92 121 141 92 121 152 49 69 152 49 69 110 32 46 120 32 46 +138 37 53 139 44 68 119 40 62 88 29 45 93 57 98 93 57 98 +58 26 51 49 25 40 0 0 0 0 0 0 0 0 0 0 0 0 +22 7 12 50 18 33 49 25 40 58 26 51 58 26 51 58 26 51 +52 25 42 49 25 40 49 25 40 40 15 29 31 10 18 28 8 14 +18 7 11 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 25 40 +93 57 98 49 25 40 73 26 42 79 28 45 118 50 79 146 108 143 +167 135 158 159 124 153 141 92 121 166 48 64 152 49 69 135 40 58 +124 37 57 121 37 56 86 29 44 105 35 55 139 44 66 124 35 54 +110 32 46 116 30 44 123 34 49 128 42 60 121 37 56 117 31 44 +133 35 50 139 38 56 105 35 55 93 57 98 137 96 131 146 108 143 +93 57 98 56 25 45 16 5 9 0 0 0 0 0 0 0 0 0 +1 0 0 30 10 18 41 14 25 39 15 26 49 25 40 52 25 42 +52 25 42 58 26 51 58 26 51 52 25 42 52 25 42 42 17 30 +50 18 33 35 12 21 18 7 11 7 2 4 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 4 6 +18 7 11 18 7 11 10 4 6 10 4 6 30 10 18 39 15 26 +30 10 18 22 7 12 30 10 18 60 26 44 76 28 47 105 35 55 +134 49 77 141 92 121 141 92 121 141 92 121 166 48 64 131 80 105 +145 47 69 145 47 69 152 49 69 141 92 121 141 92 121 139 44 68 +105 35 55 86 29 44 86 29 44 131 80 105 141 92 121 148 114 145 +148 114 145 141 92 121 166 48 64 152 49 69 105 35 55 112 32 46 +133 35 50 128 41 59 105 35 55 88 61 125 146 108 143 146 108 143 +88 61 125 58 26 51 49 25 40 22 7 12 0 0 0 0 0 0 +0 0 0 0 0 0 3 1 1 10 4 6 31 10 18 42 17 30 +48 19 31 52 25 42 58 26 51 58 26 51 52 25 42 52 25 42 +58 26 51 56 25 45 48 19 31 42 17 30 30 10 18 22 7 12 +16 5 9 7 1 3 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 +10 4 6 35 12 21 58 26 51 108 66 98 140 102 127 171 141 162 +180 150 172 171 141 162 131 80 105 76 28 47 76 28 47 105 35 55 +113 41 62 125 35 52 166 48 64 141 92 121 141 92 121 166 48 64 +166 48 64 166 48 64 152 49 69 136 44 65 152 49 69 141 92 121 +141 92 121 152 49 69 123 34 49 113 41 62 119 40 62 89 29 45 +118 50 79 120 71 102 118 50 79 123 43 68 121 37 56 105 35 55 +113 41 62 123 43 68 134 49 77 152 49 69 128 42 60 118 34 52 +129 39 59 130 43 63 105 35 55 93 63 110 146 108 143 146 108 143 +93 57 98 58 26 51 58 26 51 48 19 31 1 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 7 2 4 +22 7 12 30 10 18 35 12 21 42 17 30 52 25 42 58 26 51 +58 26 51 58 26 51 56 25 45 52 25 42 52 25 42 49 25 40 +48 19 31 42 17 30 31 10 18 20 6 10 7 2 4 7 1 3 +7 2 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 10 4 6 35 12 21 49 25 40 120 71 102 167 135 158 +191 162 183 210 187 199 234 219 224 243 232 234 241 229 231 226 205 215 +233 215 221 233 215 221 219 198 209 159 124 153 108 66 98 105 35 55 +123 43 68 152 49 69 166 48 64 162 38 49 142 34 47 123 33 48 +119 34 51 126 41 60 136 44 65 116 30 44 102 29 42 110 32 46 +123 43 68 134 49 77 131 80 105 134 49 77 134 49 77 118 50 79 +118 50 79 134 49 77 134 49 77 134 49 77 134 49 77 118 34 52 +97 28 42 69 20 33 79 27 44 97 28 42 112 32 46 128 41 59 +132 43 63 123 43 68 118 50 79 76 28 47 93 57 98 93 57 98 +62 26 49 58 26 51 58 26 51 49 25 40 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 3 1 1 1 0 0 9 3 6 39 15 26 31 10 18 +30 10 18 48 19 31 60 25 40 52 25 42 52 25 42 52 25 42 +52 25 42 52 25 42 58 24 38 52 25 42 54 21 35 50 18 33 +40 15 29 31 10 18 22 7 12 10 4 6 10 4 6 7 2 4 +3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +49 25 40 140 102 127 171 141 162 210 187 199 238 224 228 248 240 240 +243 232 234 245 236 237 245 236 237 243 232 234 234 219 224 231 213 218 +238 224 228 228 209 216 202 176 194 167 135 158 148 114 145 137 96 131 +108 54 86 113 41 62 121 37 56 130 43 63 134 49 77 138 88 121 +140 102 127 159 124 153 180 150 172 148 114 145 131 80 105 113 41 62 +105 35 55 113 41 62 134 49 77 130 43 63 145 47 69 141 92 121 +131 80 105 143 45 66 110 32 46 97 28 42 102 29 42 102 29 42 +110 32 46 119 34 51 125 35 52 128 41 59 126 41 60 128 42 60 +132 43 63 123 43 68 118 50 79 113 41 62 76 28 47 58 26 51 +58 26 51 58 26 51 58 26 51 73 26 48 30 10 18 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 7 2 4 16 5 9 22 7 12 35 12 21 42 17 30 +50 18 33 48 19 31 54 19 32 60 22 36 63 22 36 63 22 36 +58 24 38 52 24 38 58 20 33 48 16 29 47 16 28 36 11 20 +31 10 18 22 7 12 7 1 3 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 10 4 6 120 71 102 +191 162 183 210 187 199 226 205 215 238 224 228 233 215 221 234 219 224 +238 224 228 241 229 231 243 232 234 238 224 228 234 219 224 228 209 216 +233 215 221 219 198 209 191 162 183 167 135 158 148 114 145 108 66 98 +108 54 86 131 80 105 171 141 162 202 176 194 228 209 216 238 224 228 +241 229 231 245 236 237 248 240 240 248 240 240 238 224 228 219 198 209 +191 162 183 131 80 105 105 35 55 105 35 55 128 41 59 140 40 59 +144 40 57 152 40 56 139 38 56 141 40 57 141 43 61 141 42 59 +141 43 61 131 38 56 135 40 58 135 40 58 128 41 59 131 42 64 +134 49 77 118 50 79 108 54 86 108 54 86 93 57 98 93 57 98 +73 26 48 58 26 51 58 26 51 58 26 51 58 26 51 18 7 11 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 +7 1 3 7 1 3 10 4 6 16 5 9 20 6 10 41 14 25 +57 17 29 57 17 29 57 17 29 69 20 33 60 21 35 54 19 32 +57 17 29 48 19 31 41 14 25 28 8 14 16 5 9 3 1 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 62 26 49 167 135 158 +226 205 215 202 176 194 180 150 172 191 162 183 202 176 194 226 205 215 +231 213 218 231 213 218 231 213 218 219 198 209 210 187 199 202 176 194 +202 176 194 202 176 194 171 141 162 137 96 131 131 80 105 140 102 127 +210 187 199 238 222 225 243 232 234 248 240 240 248 240 240 248 240 240 +245 236 237 243 232 234 238 224 228 241 229 231 238 222 225 238 222 225 +233 215 221 210 187 199 171 141 162 131 80 105 105 35 55 131 42 64 +145 47 69 152 40 56 159 46 62 144 40 57 141 43 61 143 45 64 +140 40 59 135 40 58 130 43 63 131 42 64 131 42 64 123 43 68 +118 50 79 108 54 86 108 54 86 93 57 98 76 28 47 76 28 47 +76 28 47 58 26 51 58 26 51 58 26 51 58 26 51 30 10 18 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 +7 1 3 20 6 10 31 10 18 41 13 21 57 17 29 57 17 29 +69 20 33 69 20 33 79 27 44 71 23 37 71 23 37 57 17 29 +36 11 20 38 12 21 16 5 9 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 140 102 127 +210 187 199 226 205 215 210 187 199 210 187 199 210 187 199 210 187 199 +210 187 199 210 187 199 202 176 194 202 176 194 180 150 172 191 162 183 +180 150 172 171 141 162 146 108 143 95 59 101 196 169 185 245 236 237 +238 224 228 234 219 224 238 224 228 238 224 228 243 232 234 243 232 234 +245 236 237 245 236 237 238 224 228 231 213 218 238 224 228 226 205 215 +219 198 209 202 176 194 196 169 185 167 135 158 120 71 102 105 35 55 +134 49 77 152 49 69 159 46 62 159 46 63 152 40 56 152 40 56 +139 44 66 134 49 77 134 49 77 134 49 77 118 50 79 118 50 79 +108 54 86 93 57 98 73 26 48 93 57 98 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 93 57 98 49 25 40 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 3 1 1 10 4 6 7 2 4 +10 4 6 28 8 14 57 17 29 97 28 42 102 29 42 97 28 42 +86 29 44 69 20 33 69 20 33 41 13 21 20 6 10 16 5 9 +9 3 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 25 40 +196 169 185 228 209 216 226 205 215 233 215 221 234 219 224 238 224 228 +238 224 228 234 219 224 233 215 221 231 213 218 202 176 194 180 150 172 +171 141 162 146 108 143 137 96 131 196 169 185 238 224 228 243 232 234 +241 229 231 234 219 224 233 215 221 234 219 224 241 229 231 234 219 224 +238 224 228 234 219 224 228 209 216 219 198 209 191 162 183 191 162 183 +202 176 194 191 162 183 159 124 153 148 114 145 137 96 131 93 57 98 +113 41 62 134 49 77 145 47 69 145 47 69 134 49 77 134 49 77 +134 49 77 118 50 79 108 54 86 108 54 86 108 54 86 76 28 47 +73 26 48 73 26 48 73 26 48 58 26 51 58 26 51 58 26 51 +58 26 51 93 57 98 93 57 98 93 57 98 88 61 125 58 26 51 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 4 0 1 36 11 20 69 20 33 +102 29 42 117 31 42 102 29 42 71 23 37 71 23 37 61 21 34 +41 13 21 9 3 6 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 1 +67 27 47 167 135 158 210 187 199 219 198 209 226 205 215 234 219 224 +226 205 215 233 215 221 234 219 224 226 205 215 219 198 209 180 150 172 +167 135 158 137 96 131 137 96 131 226 205 215 210 187 199 202 176 194 +219 198 209 226 205 215 226 205 215 219 198 209 228 209 216 226 205 215 +210 187 199 202 176 194 202 176 194 191 162 183 171 141 162 167 135 158 +159 124 153 159 124 153 137 96 131 137 96 131 88 61 125 93 63 110 +76 28 47 118 50 79 118 50 79 118 50 79 108 54 86 108 54 86 +108 54 86 108 54 86 76 28 47 73 26 48 76 28 47 73 26 48 +58 26 51 58 26 51 58 26 51 58 26 51 93 57 98 88 61 125 +88 61 125 88 61 125 88 61 125 93 57 98 58 26 51 49 25 40 +3 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 41 13 21 102 29 42 132 30 39 97 28 42 69 20 33 +71 23 37 47 16 28 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 18 7 11 93 57 98 108 54 86 140 102 127 167 135 158 +171 141 162 167 135 158 159 124 153 159 124 153 159 124 153 146 108 143 +137 96 131 93 63 110 93 57 98 219 198 209 231 213 218 219 198 209 +202 176 194 191 162 183 171 141 162 167 135 158 171 141 162 167 135 158 +159 124 153 159 124 153 148 114 145 146 108 143 146 108 143 146 108 143 +137 96 131 137 96 131 88 61 125 88 61 125 88 61 125 93 57 98 +73 26 48 105 35 55 105 35 55 76 28 47 73 26 48 73 26 48 +76 28 47 58 26 51 58 26 51 93 57 98 93 57 98 93 57 98 +88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 93 57 98 93 57 98 40 15 29 16 5 9 1 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 4 0 1 69 20 33 132 30 39 117 31 42 +86 29 44 69 20 33 9 3 6 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 4 6 +58 26 51 140 102 127 140 102 127 219 198 209 238 224 228 234 219 224 +226 205 215 226 205 215 219 198 209 202 176 194 196 169 185 171 141 162 +159 124 153 146 108 143 138 88 121 108 66 98 93 63 110 93 63 110 +93 63 110 93 63 110 88 61 125 88 61 125 88 61 125 88 61 125 +93 57 98 93 57 98 93 57 98 93 57 98 93 57 98 93 57 98 +93 57 98 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +58 26 51 40 15 29 7 2 4 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 102 29 42 102 29 42 +71 23 37 79 27 44 38 12 21 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 3 1 1 167 135 158 202 176 194 210 187 199 +228 209 216 228 209 216 228 209 216 228 209 216 226 205 215 219 198 209 +210 187 199 191 162 183 180 150 172 148 114 145 137 96 131 137 96 131 +88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 88 61 125 58 26 51 40 15 29 16 5 9 3 1 1 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 57 17 29 97 28 42 +97 28 42 86 29 44 41 14 25 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 7 2 4 52 25 42 131 80 105 +148 114 145 196 169 185 219 198 209 226 205 215 233 215 221 219 198 209 +202 176 194 202 176 194 191 162 183 171 141 162 148 114 145 146 108 143 +137 96 131 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 88 61 125 +88 61 125 88 61 125 93 57 98 58 26 51 58 26 51 49 25 40 +18 7 11 9 3 6 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 16 5 9 69 20 33 28 8 14 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 7 1 3 97 28 42 97 28 42 +71 23 37 61 21 34 36 11 20 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 22 7 12 52 25 42 67 27 47 93 57 98 93 57 98 +93 57 98 93 57 98 58 26 51 93 57 98 58 26 51 58 26 51 +58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 58 26 51 +49 25 40 40 15 29 40 15 29 16 5 9 9 3 6 3 1 1 +1 0 0 4 0 1 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 41 13 21 148 30 39 185 38 47 152 32 42 28 8 14 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 20 6 10 86 29 44 97 28 42 69 20 33 +47 16 28 40 15 29 30 10 18 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 28 8 14 +97 28 42 185 38 47 185 38 47 159 35 49 152 32 42 102 29 42 +28 8 14 7 2 4 0 0 0 1 0 0 7 1 3 28 8 14 +31 10 18 57 17 29 97 28 42 69 20 33 54 19 32 48 19 31 +28 8 14 9 3 6 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 16 5 9 69 20 33 121 30 42 152 32 42 +185 38 47 152 32 42 152 32 42 152 32 42 141 31 41 132 30 39 +97 28 42 69 20 33 57 17 29 69 20 33 69 20 33 71 23 37 +71 23 37 71 23 37 51 18 32 58 20 33 48 19 31 18 7 11 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 10 4 6 39 15 26 +54 19 32 97 28 42 148 30 39 185 38 47 185 38 47 185 38 47 +159 35 49 152 32 42 152 32 42 148 30 39 142 31 41 141 31 41 +132 30 39 117 31 42 97 28 42 73 26 42 67 22 35 58 20 33 +47 16 28 47 16 28 40 15 29 40 15 29 22 7 12 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 10 4 6 47 16 28 69 20 33 131 32 43 +152 32 42 185 38 47 185 38 47 185 38 47 185 38 47 159 35 49 +148 30 39 148 30 39 152 32 42 141 31 41 148 30 39 152 32 42 +138 32 45 102 29 42 71 23 37 52 24 38 48 19 31 47 16 28 +41 14 25 41 14 25 41 14 25 41 13 21 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +7 1 3 102 29 42 148 30 39 185 38 47 185 38 47 185 38 47 +162 38 49 152 32 42 152 32 42 148 30 39 148 30 39 132 30 39 +124 30 42 117 31 42 117 31 42 97 28 42 97 28 42 86 29 44 +71 23 37 47 16 28 36 11 20 41 13 21 28 8 14 28 8 14 +28 8 14 22 7 12 16 5 9 16 5 9 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +22 7 12 117 31 42 102 29 42 97 28 42 69 20 33 69 20 33 +57 17 29 47 16 28 47 16 28 41 13 21 41 13 21 41 13 21 +41 13 21 41 13 21 36 11 20 41 13 21 30 10 18 7 2 4 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 diff --git b/drivers/video/logo/logo_debian_clut224.ppm b/drivers/video/logo/logo_debian_clut224.ppm new file mode 100644 index 0000000..0daf773 --- /dev/null +++ b/drivers/video/logo/logo_debian_clut224.ppm @@ -0,0 +1,883 @@ +P3 +64 80 +255 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 36 1 13 99 3 37 93 2 34 135 4 50 95 3 36 +41 1 15 21 0 7 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 64 2 24 +154 4 57 213 6 80 228 7 85 227 7 85 226 7 85 209 6 78 +217 7 81 194 6 73 98 3 37 95 3 36 126 4 47 159 4 59 +147 4 55 150 4 56 175 4 65 106 3 39 43 1 16 2 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 9 0 3 87 2 32 177 5 66 224 7 84 +225 7 85 218 7 82 215 7 81 215 7 81 214 6 80 217 7 82 +217 7 82 219 7 82 226 7 85 227 7 85 226 7 85 225 7 84 +225 7 85 224 7 84 223 7 84 227 7 85 219 7 82 182 5 68 +106 3 39 24 0 9 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +4 0 1 90 2 33 191 6 71 227 7 85 223 7 84 216 7 81 +214 7 80 214 7 81 215 7 81 215 7 81 215 7 81 215 7 81 +214 7 80 215 7 81 215 7 81 215 7 81 215 7 81 214 7 81 +215 7 81 215 7 81 214 7 80 215 7 81 217 7 81 222 7 83 +228 7 86 206 6 77 120 3 45 41 1 15 63 1 23 104 3 39 +49 1 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 62 1 23 +182 5 68 228 7 85 221 7 83 215 7 81 214 7 80 215 7 81 +215 7 81 215 7 81 215 7 81 215 7 81 215 7 81 215 7 81 +215 7 81 215 7 81 215 7 81 215 7 81 215 7 81 215 7 81 +215 7 81 215 7 81 215 7 81 215 7 81 215 7 81 214 7 80 +215 7 81 219 7 82 226 7 85 219 7 82 223 7 84 231 7 86 +215 7 80 40 1 15 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 9 0 3 133 4 50 225 7 84 +223 7 84 215 7 81 214 7 80 215 7 81 215 7 81 215 7 81 +215 7 81 215 7 81 215 7 81 214 7 80 215 7 81 216 7 81 +216 7 81 216 7 81 217 7 81 218 7 82 218 7 82 218 7 82 +217 7 82 216 7 81 215 7 81 214 7 81 214 7 81 215 7 81 +215 7 81 214 7 80 215 7 81 217 7 81 216 7 81 214 7 80 +221 7 83 204 6 76 81 2 30 57 1 21 2 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 1 0 0 99 3 37 171 4 64 229 7 86 216 7 81 +214 7 80 214 7 81 215 7 81 215 7 81 215 7 81 215 7 81 +214 7 81 214 7 80 217 7 81 224 7 84 227 7 85 222 7 83 +219 7 82 222 7 83 210 5 78 205 6 77 202 5 76 205 6 77 +209 6 78 219 7 82 225 7 84 227 7 85 224 7 84 219 7 82 +215 7 81 214 7 80 215 7 81 215 7 81 215 7 81 214 7 81 +214 7 80 220 7 82 221 7 83 217 7 81 166 5 62 26 0 9 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +6 0 2 148 4 55 232 7 87 222 7 83 214 7 80 214 7 80 +215 7 81 215 7 81 215 7 81 215 7 81 215 7 81 214 7 80 +215 7 81 225 7 84 214 6 80 153 4 56 95 3 36 54 1 20 +42 1 16 51 1 19 26 0 10 16 0 6 14 0 5 16 0 6 +24 0 9 41 1 15 68 1 25 105 2 39 155 4 58 199 6 74 +225 7 84 225 7 84 217 7 81 214 7 80 214 7 81 215 7 81 +215 7 81 214 7 80 216 7 81 217 7 81 227 7 85 201 6 75 +54 1 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 0 5 +166 5 62 229 7 86 215 7 81 214 7 81 214 7 80 215 7 81 +215 7 81 215 7 81 215 7 81 214 7 80 217 7 82 225 7 85 +225 7 84 160 4 59 42 1 15 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 4 +68 1 25 151 4 56 215 7 80 225 7 84 215 7 81 214 7 80 +215 7 81 215 7 81 215 7 81 214 7 80 214 7 80 222 7 83 +215 6 81 71 2 26 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 22 0 8 180 5 67 +228 7 85 214 7 80 214 7 80 215 7 81 215 7 81 215 7 81 +214 7 80 215 7 81 219 7 82 227 7 85 207 6 77 147 4 55 +64 2 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 43 1 16 159 5 60 227 7 85 218 7 82 +214 7 80 215 7 81 215 7 81 215 7 81 214 7 81 214 7 80 +220 7 82 222 7 83 74 2 28 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 22 0 8 183 5 68 227 7 85 +214 7 80 214 7 80 215 7 81 215 7 81 215 7 81 214 7 80 +217 7 81 225 7 85 195 5 73 163 5 61 90 2 33 10 0 4 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 97 3 36 215 5 80 +221 7 83 214 7 80 214 7 81 215 7 81 215 7 81 215 7 81 +214 7 80 220 7 82 216 7 80 55 1 20 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 10 0 3 175 4 65 227 7 85 214 7 80 +214 7 80 215 7 81 215 7 81 215 7 81 214 7 80 220 7 82 +214 5 80 105 3 39 13 0 5 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 57 1 21 +204 6 76 222 7 83 214 7 80 214 7 81 215 7 81 215 7 81 +215 7 81 214 7 80 222 7 83 201 6 75 29 0 10 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 124 4 46 230 7 86 214 7 80 214 7 80 +215 7 81 215 7 81 214 7 80 215 7 81 222 7 83 206 5 76 +58 1 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +41 1 15 201 6 75 222 7 83 214 7 80 215 7 81 215 7 81 +215 7 81 214 7 81 214 7 80 226 7 85 170 5 63 3 0 1 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 43 1 16 211 6 79 218 7 81 214 7 80 215 7 81 +214 7 81 214 7 80 217 7 81 226 7 85 206 6 77 105 3 39 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 42 1 15 206 6 77 220 7 82 214 7 80 215 7 81 +215 7 81 215 7 81 214 7 80 215 7 80 229 7 86 105 3 39 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +5 0 1 183 5 68 224 7 84 214 7 80 215 7 81 214 7 80 +214 7 80 222 7 83 214 5 80 115 3 43 19 0 7 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 62 1 23 219 6 82 217 7 81 214 7 80 +215 7 81 215 7 81 215 7 81 214 7 80 219 7 82 211 6 79 +30 1 11 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +69 1 26 226 7 85 215 7 81 214 7 81 214 7 80 214 7 80 +226 7 85 191 6 71 49 1 18 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 104 3 39 228 7 85 214 7 80 +214 7 81 215 7 81 215 7 81 215 7 81 214 7 80 227 7 85 +138 3 51 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 51 1 19 0 0 0 0 0 0 0 0 0 +162 4 60 224 7 84 214 7 80 214 7 80 214 7 80 227 7 85 +170 5 64 20 0 7 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 166 5 62 224 7 84 +214 7 80 215 7 81 215 7 81 214 7 80 215 7 81 220 7 82 +214 6 80 32 1 12 0 0 0 0 0 0 +0 0 0 5 0 2 49 1 18 0 0 0 0 0 0 40 1 15 +218 6 81 216 7 81 214 7 80 214 7 80 228 7 85 163 5 61 +8 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 37 1 14 214 6 80 +216 7 81 214 7 81 214 7 81 214 7 80 222 7 83 215 7 81 +228 7 85 106 3 39 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 32 1 12 174 5 65 +222 7 83 214 7 80 214 7 80 227 7 85 163 5 61 8 0 3 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 129 4 48 +226 7 85 214 7 80 215 7 81 222 7 83 115 3 43 44 1 16 +140 3 52 85 2 31 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 15 0 5 201 6 75 225 7 84 +214 7 80 214 7 80 225 7 84 178 4 66 12 0 4 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 26 0 9 +210 5 78 217 7 81 215 7 81 227 7 85 79 2 30 0 0 0 +36 1 13 82 2 31 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 97 3 36 229 7 86 215 7 80 +214 7 80 221 7 83 199 5 74 25 0 9 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +145 4 54 224 7 84 214 7 80 222 7 83 168 5 62 40 1 15 +0 0 0 20 0 7 0 0 0 0 0 0 +0 0 0 0 0 0 4 0 1 185 6 69 221 7 83 214 7 80 +216 7 81 213 6 80 54 1 20 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 40 1 15 110 3 41 163 5 61 188 6 70 +194 6 73 182 5 68 145 4 54 80 2 30 13 0 5 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +64 2 24 224 7 84 215 7 80 214 7 80 226 7 84 188 6 70 +3 0 1 0 0 0 0 0 0 0 0 0 +6 0 2 0 0 0 62 1 23 225 7 84 215 7 81 215 7 81 +223 7 83 126 4 47 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +43 1 16 160 4 59 230 7 86 233 7 87 193 6 72 147 4 55 +117 3 44 106 3 39 120 3 45 150 4 56 159 4 59 90 2 33 +9 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +12 0 4 201 6 75 218 7 81 214 7 80 218 7 81 213 6 80 +25 0 9 0 0 0 0 0 0 0 0 0 +33 1 12 0 0 0 145 4 54 226 7 84 214 7 80 214 7 80 +224 7 84 154 4 57 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 95 3 36 +225 7 84 226 7 85 140 3 52 50 1 18 4 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 8 0 3 56 1 21 +76 2 28 17 0 6 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 163 5 61 222 7 83 215 7 81 219 7 82 183 5 68 +51 1 19 0 0 0 0 0 0 0 0 0 +1 0 0 17 0 6 204 6 76 219 7 82 215 7 81 215 7 80 +224 7 84 71 2 26 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 102 2 37 239 7 89 +175 4 65 44 1 16 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 14 0 5 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 131 4 49 225 7 84 215 7 81 224 7 84 142 4 53 +8 0 3 0 0 0 0 0 0 0 0 0 +0 0 0 80 2 30 226 7 85 215 7 80 214 7 80 222 7 83 +172 4 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 50 1 19 235 7 88 148 4 55 +7 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 97 3 36 224 7 84 215 7 81 219 7 82 202 6 75 +20 0 7 0 0 0 0 0 0 0 0 0 +0 0 0 151 4 56 225 7 84 214 7 80 215 7 80 222 7 83 +62 1 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 25 0 9 195 5 73 162 4 60 2 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 53 1 20 220 7 82 215 7 81 215 7 80 227 7 85 +87 2 32 0 0 0 0 0 0 0 0 0 +15 0 6 202 5 76 219 7 82 214 7 81 222 7 83 166 5 62 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 3 0 1 175 4 65 207 6 77 15 0 5 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 56 1 21 221 7 83 214 7 81 216 7 80 223 7 83 +58 1 21 0 0 0 0 0 0 0 0 0 +64 2 24 224 7 84 215 7 80 215 7 81 223 7 84 77 2 29 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 71 1 26 239 7 89 80 2 30 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 50 1 18 219 7 82 215 7 81 215 7 81 225 7 84 +76 2 28 0 0 0 0 0 0 0 0 0 +52 1 19 221 7 82 216 7 80 215 7 80 212 6 80 32 1 12 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 160 4 59 193 6 72 1 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 43 1 16 216 7 81 216 7 81 214 7 81 220 7 82 +182 5 68 22 0 8 0 0 0 0 0 0 +33 1 12 212 6 80 217 7 81 217 7 81 204 6 76 19 0 7 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +20 0 7 220 5 82 111 3 41 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 43 1 16 216 7 81 216 7 81 214 7 81 221 7 83 +186 5 70 6 0 2 0 0 0 0 0 0 +32 1 12 212 6 80 217 7 81 218 7 82 194 6 73 7 0 2 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +117 3 44 232 7 87 47 1 17 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 41 1 15 36 1 13 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 48 1 18 218 7 82 215 7 81 214 7 80 231 7 86 +111 3 41 0 0 0 0 0 0 0 0 0 +41 1 15 218 7 82 216 7 80 221 7 83 177 5 66 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +180 5 67 211 6 79 16 0 6 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 15 0 5 16 0 6 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 69 1 26 223 7 83 214 6 80 220 7 82 188 6 70 +47 1 17 0 0 0 0 0 0 0 0 0 +41 1 15 217 7 81 217 7 81 223 7 83 157 5 59 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 1 +193 5 72 195 5 73 4 0 1 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 56 1 21 215 6 81 216 6 81 224 7 84 104 3 39 +0 0 0 0 0 0 0 0 0 0 0 0 +51 1 19 221 7 82 216 7 81 224 7 83 135 4 50 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 0 2 +199 5 74 193 6 72 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 33 0 12 212 6 80 218 7 82 217 7 81 36 1 13 +0 0 0 0 0 0 0 0 0 0 0 0 +53 1 20 222 7 83 215 7 81 225 7 84 115 3 43 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 2 +193 5 72 221 7 82 62 1 23 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 71 1 26 223 7 83 219 7 82 205 6 76 17 0 6 +0 0 0 0 0 0 0 0 0 0 0 0 +53 1 20 222 7 83 215 7 81 225 7 84 97 3 36 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +182 5 68 233 7 87 97 3 36 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 142 4 53 224 7 84 224 7 84 160 4 59 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +42 1 16 218 7 82 216 7 81 224 7 83 82 2 31 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +157 5 59 233 7 88 133 4 50 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +13 0 5 206 6 77 218 7 82 226 7 85 71 2 26 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +44 1 16 219 7 82 216 7 81 223 7 84 79 2 30 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +112 3 42 233 7 87 185 6 69 2 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 26 0 10 7 0 2 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 8 +140 3 52 222 7 83 222 7 83 177 5 66 3 0 1 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +54 1 20 222 7 83 215 7 81 223 7 83 79 2 30 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +48 1 18 223 7 84 223 7 83 53 1 20 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 159 5 60 +231 7 86 218 7 82 234 7 87 80 2 30 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +50 1 18 221 7 82 216 7 80 224 7 84 82 2 31 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 174 5 65 233 7 87 150 4 56 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 10 0 3 18 0 6 19 0 7 9 0 3 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 180 5 67 +225 7 84 199 6 74 110 3 41 20 0 7 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +41 1 15 217 7 81 217 7 81 225 7 84 106 3 39 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 68 1 25 228 7 85 222 7 83 56 1 21 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 21 0 8 87 2 32 117 3 44 19 0 7 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 43 1 16 207 6 77 +229 7 86 98 2 36 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +30 1 11 211 6 79 217 7 81 223 7 83 151 4 56 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 148 4 55 235 7 88 188 6 70 13 0 4 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 60 1 22 207 6 77 225 7 84 +221 7 83 34 0 12 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +15 0 6 201 6 75 219 7 82 216 7 81 209 6 78 36 1 13 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 53 1 20 +10 0 3 0 0 0 15 0 5 188 6 70 236 7 88 158 4 59 +4 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 80 2 30 215 7 80 226 7 84 215 5 80 +76 2 28 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +1 0 0 186 5 70 221 7 83 214 7 80 225 7 84 77 2 29 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 22 0 8 +55 1 20 7 0 2 0 0 0 33 0 12 202 6 75 237 7 89 +151 4 56 7 0 2 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 111 3 41 225 7 84 225 7 84 215 5 80 67 2 25 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 159 5 60 224 7 84 214 7 81 221 7 83 53 1 20 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +36 1 13 26 0 9 39 1 14 0 0 0 36 1 13 193 5 72 +238 7 89 174 5 65 36 1 13 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 16 0 6 75 2 28 +154 4 57 228 7 85 228 7 85 204 6 76 57 1 21 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 120 3 45 227 7 85 214 7 80 223 7 83 104 3 39 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 5 0 2 15 0 5 37 1 14 0 0 0 22 0 8 +172 4 64 238 7 89 211 6 79 106 3 39 14 0 5 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 50 1 18 182 5 68 230 7 86 +228 7 86 229 7 86 170 5 63 30 1 11 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 74 2 28 225 7 84 215 7 80 223 7 83 159 4 59 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 9 0 3 29 0 10 34 0 12 +3 0 1 135 4 50 230 7 86 232 7 87 199 6 74 126 4 47 +56 1 21 19 0 7 3 0 1 0 0 0 3 0 1 21 0 8 +67 2 25 131 4 49 170 5 63 221 7 83 228 7 86 228 7 86 +204 6 76 99 3 37 1 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 29 0 10 211 6 79 217 7 81 220 7 82 182 5 68 +2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 75 2 28 +126 4 47 22 0 8 98 2 36 205 6 77 230 7 86 238 7 89 +232 7 87 209 6 78 191 6 71 188 6 70 192 6 72 209 6 78 +230 7 86 235 7 88 235 7 88 226 7 85 193 6 72 112 3 42 +21 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 177 5 66 222 7 83 215 7 80 221 7 83 +57 1 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +54 1 20 140 3 52 133 3 49 47 1 17 68 1 25 126 4 47 +171 4 64 201 6 75 216 7 81 221 7 83 221 7 83 212 6 80 +199 6 74 172 4 64 126 4 47 64 2 24 10 0 3 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 117 3 44 227 7 85 214 7 80 221 7 83 +170 5 63 8 0 3 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 1 0 0 71 2 26 117 3 44 53 1 20 9 0 3 +0 0 0 6 0 2 19 0 7 26 0 10 27 0 10 24 0 9 +12 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 49 1 18 222 7 83 216 7 81 214 7 80 +220 7 82 189 6 71 133 4 50 12 0 4 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 34 0 12 64 2 24 +76 2 28 71 1 26 60 1 22 47 1 17 40 1 15 8 0 3 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 2 0 0 181 5 67 222 7 83 214 7 80 +214 7 81 221 7 83 233 7 88 140 3 52 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0 12 0 4 21 0 8 22 0 8 16 0 6 2 0 0 +0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 +0 0 0