Temporal-Difference learning Sample Clauses

Temporal-Difference learning. ‌ Temporal-Difference (TD) learning is a significant and novel idea to RL, which combines Monte Carlo ideas and dynamic programming ideas [110]. Using Monte Carlo ideas, TD methods can learn directly from raw experience without prior knowledge of the environment’s dynamics. Using dynamic programming ideas, TD methods can continuously update estimates without waiting for a final outcome. TD methods are usually applied to the policy evaluation, prediction problem or control problem. Before introducing TD methods, we first review the simplest every-visit Monte Carlo method for non-stationary environment, expressed as where Gt is the actual return following time t, and α is a constant step size. Note that the return Gt is only revealed at the end of an episode, thus Monte Carlo methods update must wait until the end. However, TD methods can update estimates each step of the episode. Given some experience following a policy π, TD methods update the estimate V of the value function vπ for the state St occurring in that experience. At time t + 1, the update of the simplest TD method is given by where Rt+1 is the observed reward, γ is a constant discount factor. This methods is termed as TD(0) or one-step TD, because it is a special case of the TD(λ) or n-step TD methods developed in [88]. TD methods update estimates based on existing estimates V (St+1), which are called as bootstrapping, i.e., learn a guess from a guess. For the control problem, the alternating sequence of states and state-action pairs in an episode are usually described in Fig. 2.4, as follows Figure 2.4: An alternating sequence of states and state-action pairs. Many RL algorithms using TD methods have been developed to solve the control problem, e.g., Sarsa (on-policy TD control) in [109] and Q-learning (off- policy TD control) in [125]. Both of Sarsa and Q-learning algorithms attempts to learn an action-value function rather than a state-value function. Take Q-learning as an example, the update of Q-learning is defined by Q(St, At) ← Q(St, At) + α Rt+1 + γ max Q(St+1, a) − Q(St, At) , (2.8) where Q(·) is the action-value function. Q-learning algorithm has been proved that it converges with probability 1 to the optimal action-values [110]. The generalized Q-learning algorithm is depicted in the following Algorithm 2.1. Algorithm 2.1 Q-learning (off-policy TD control)
AutoNDA by SimpleDocs
Temporal-Difference learning. Commented [D(15]: can this be left out?

Related to Temporal-Difference learning

  • Bilingual Differential When formally assigned in the employee’s position description, an employee assigned to interpret to or from another language to English will receive a differential of five percent (5%) of base pay.

  • Cost of Living Adjustments Effective December 1, 2021, Compensation Plan salary rates shall be increased by two and five tenths percent (2.5%) but not less than eighty-five dollars ($85) per month (prorated for part-time employees). Effective December 1, 2022, Compensation Plan salary rates shall be increased by three and one tenth percent (3.1%) but not less than one hundred dollars ($100) per month (prorated for part-time employees). (See Appendix C & E.)

  • Cost of Living Adjustment For each year following the Initial Term, unless the parties shall otherwise agree and provided that the service mix and volumes remain consistent as previously provided in the Initial Term, the total fee for all services shall equal the fee that would be charged for the same services based on a fee rate (as reflected in a fee rate schedule) increased by the percentage increase for the twelve-month period of such previous calendar year of the CPI-W (defined below) or, in the event that publication of such index is terminated, any successor or substitute index, appropriately adjusted, acceptable to both parties. As used herein, “CPI-W” shall mean the Consumer Price Index for Urban Wage Earners and Clerical Workers (Area: Boston-Brockton-Nashua, MA-NH-ME-CT; Base Period: 1982-84=100), as published by the United States Department of Labor, Bureau of Labor Statistics.

  • Inability to Determine Eurodollar Rate In the event, prior to the commencement of any Interest Period relating to any Eurodollar Rate Loan, the Administrative Agent shall determine or be notified by the Required Lenders that adequate and reasonable methods do not exist for ascertaining the Eurodollar Rate that would otherwise determine the rate of interest to be applicable to any Eurodollar Rate Loan during any Interest Period, the Administrative Agent shall forthwith give notice of such determination (which shall be conclusive and binding on the Borrower and the Lenders) to the Borrower and the Lenders. In such event (a) any Loan Request or Conversion Request with respect to Eurodollar Rate Loans shall be automatically withdrawn and shall be deemed a request for Base Rate Loans, (b) each Eurodollar Rate Loan will automatically, on the last day of the then current Interest Period relating thereto, become a Base Rate Loan, and (c) the obligations of the Lenders to make Eurodollar Rate Loans shall be suspended until the Administrative Agent or the Required Lenders determine that the circumstances giving rise to such suspension no longer exist, whereupon the Administrative Agent or, as the case may be, the Administrative Agent upon the instruction of the Required Lenders, shall so notify the Borrower and the Lenders.

  • Night Shift Differential Unit 12 employees who regularly work shifts shall receive a night shift differential as set forth below: A. Employees shall qualify for the first night shift pay differential of forty (40) cents per hour where four (4) or more hours of the regularly scheduled work shift falls between 6 p.m. and 12 midnight. B. Employees shall qualify for the second night shift pay differential of fifty (50) cents per hour where four (4) or more hours of the regularly scheduled work shift fall between 12 midnight and 6 a.m. C. A "regularly scheduled work shift" are those regularly assigned work hours established by the department director or designee.

  • Supervisory Differential Adjustment The Appointing Officer shall adjust the compensation of a supervisory employee whose compensation grade is set herein subject to the following conditions:

  • Weekend Differential Employees assigned to State institutions other than Maine State Prison shall be eligible for a weekend differential of fifty cents ($.50) per hour to the base for shifts beginning between 10:00 p.m. Friday and 9:59 p.m.

  • Increased Cost and Reduced Return; Capital Adequacy; Reserves on Eurodollar Rate Loans (a) If any Lender determines that as a result of the introduction of or any change in or in the interpretation of any Law, or such Lender’s compliance therewith, there shall be any increase in the cost to such Lender of agreeing to make or making, funding or maintaining Eurodollar Rate Loans or (as the case may be) issuing or participating in Letters of Credit, or a reduction in the amount received or receivable by such Lender in connection with any of the foregoing (excluding for purposes of this subsection (a) any such increased costs or reduction in amount resulting from (i) Taxes or Other Taxes (as to which Section 3.01 shall govern), (ii) changes in the basis of taxation of overall net income or overall gross income by the United States or any foreign jurisdiction or any political subdivision of either thereof under the Laws of which such Lender is organized or has its Lending Office, and (iii) reserve requirements contemplated by Section 3.04(c)), then from time to time upon demand of such Lender (with a copy of such demand to the Administrative Agent), the Borrower shall pay to such Lender such additional amounts as will compensate such Lender for such increased cost or reduction. (b) If any Lender determines that the introduction of any Law regarding capital adequacy or any change therein or in the interpretation thereof, or compliance by such Lender (or its Lending Office) therewith, has the effect of reducing the rate of return on the capital of such Lender or any corporation controlling such Lender as a consequence of such Lender’s obligations hereunder (taking into consideration its policies with respect to capital adequacy and such Lender’s desired return on capital), then from time to time upon demand of such Lender (with a copy of such demand to the Administrative Agent), the Borrower shall pay to such Lender such additional amounts as will compensate such Lender for such reduction. (c) The Borrower shall pay to each Lender, as long as such Lender shall be required to maintain reserves with respect to liabilities or assets consisting of or including Eurocurrency funds or deposits (currently known as “Eurocurrency liabilities”), additional interest on the unpaid principal amount of each Eurodollar Rate Loan equal to the actual costs of such reserves allocated to such Loan by such Lender (as determined by such Lender in good faith, which determination shall be conclusive), which shall be due and payable on each date on which interest is payable on such Loan, provided the Borrower shall have received at least 15 days’ prior notice (with a copy to the Administrative Agent) of such additional interest from such Lender. If a Lender fails to give notice 15 days prior to the relevant Interest Payment Date, such additional interest shall be due and payable 15 days from receipt of such notice.

  • Shift Differential A. Shift differential will be $.60 cents per hour. B. Employees eligible for shift differential are those whose work shift begins before 6:00 a.m. or ends on or after 7:00 p.m. and are scheduled by their supervisor for a total shift of at least six (6) hours in duration. This shift differential shall not apply to those employees who have requested and have been granted flexible work scheduling.

  • Sustainability Adjustments (a) DEI may deliver a Pricing Certificate to the Administrative Agent in respect of the most recently ended calendar year on any date prior to the date that is 120 days following the last day of such calendar year (the date the Administrative Agent’s receipt thereof, each a “Pricing Certificate Date”), which DEI may or may not do, in its sole discretion. If DEI so delivers a Pricing Certificate in respect of a calendar year, (i) the Applicable Percentage for the Revolving Loans incurred by DEI shall be increased or decreased (or neither increased nor decreased), as applicable, pursuant to the Sustainability Margin Adjustment as set forth in the KPI Metrics Certificate delivered with such Pricing Certificate, and (ii) the Applicable Percentage for the Facility Fee for Commitments under the DEI Sublimit shall be increased or decreased (or neither increased nor decreased), as applicable, pursuant to the Sustainability Fee Adjustment as set forth in such KPI Metrics Certificate. If no Pricing Certificate is so delivered in respect of a calendar year, the Sustainability Margin Adjustment and the Sustainability Fee Adjustment in respect of such calendar year shall be determined pursuant to Section 1.7(c). For purposes of the foregoing, (A) if a Pricing Certificate is so delivered for any calendar year, the Sustainability Margin Adjustment and the Sustainability Fee Adjustment shall be determined as of the fifth Business Day following the Pricing Certificate Date for such Pricing Certificate based upon the KPI Metrics for such calendar year set forth in the KPI Metrics Certificate delivered with such Pricing Certificate and the calculations of the Sustainability Margin Adjustment and the Sustainability Fee Adjustment in such KPI Metrics Certificate and (B) if no Pricing Certificate is so delivered in respect of such calendar year, the Sustainability Margin Adjustment and the Sustainability Fee Adjustment shall be determined pursuant to Section 1.7(c) effective as of the Business Day immediately following the date that is 120 days following the last day of such calendar year (such fifth (5th) Business Day or such Business Day, as applicable, each a “Sustainability Pricing Adjustment Date”). Each change in the Applicable Percentages on any Sustainability Pricing Adjustment Date shall be effective during the period commencing on and including such Sustainability Pricing Adjustment Date and ending on the date immediately preceding the next Sustainability Pricing Adjustment Date. (b) For the avoidance of doubt, only one Pricing Certificate (or, in the case of non-delivery of a Pricing Certificate, zero Pricing Certificates) may be delivered in respect of any calendar year. It is further understood and agreed that the Applicable Percentage for Revolving Loans incurred by DEI will never be reduced or increased by more than 0.05% and that the Applicable Percentage for the Facility Fee for Commitments under the DEI Sublimit will never be reduced or increased by more than 0.01%, pursuant to the Sustainability Margin Adjustment and the Sustainability Fee Adjustment, respectively, on any Sustainability Pricing Adjustment Date. For the avoidance of doubt, any adjustment to the Applicable Percentages for such Revolving Loans or such Facility Fee by reason of meeting one or several KPI Metrics in any calendar year shall not be cumulative year-over-year. The adjustments pursuant to this Section made on any Sustainability Pricing Adjustment Date shall only apply for the period until the date immediately preceding the next Sustainability Pricing Adjustment Date. (c) It is hereby understood and agreed that if no such Pricing Certificate with respect to a calendar year is delivered by DEI within the period set forth in this Section 1.7, the Sustainability Margin Adjustment will be positive 0.05% and the Sustainability Fee Adjustment will be positive 0.01% commencing on the last day of such period and continuing until the day immediately prior to the next Sustainability Pricing Adjustment Date. (d) If (i)(A) a Borrower or any Lender becomes aware of any material inaccuracy in the Sustainability Margin Adjustment, the Sustainability Fee Adjustment or the KPI Metrics as reported in a Pricing Certificate (any such material inaccuracy, a “Pricing Certificate Inaccuracy”) and, in the case of any Lender, such Lender delivers, not later than 10 Business Days after obtaining knowledge thereof, a written notice to the Administrative Agent describing such Pricing Certificate Inaccuracy in reasonable detail (which description shall be shared with each Lender and the Borrowers), or (B) the Borrowers and the Lenders agree that there was a Pricing Certificate Inaccuracy at the time of delivery of a Pricing Certificate, and (ii) a proper calculation of the Sustainability Margin Adjustment, Sustainability Fee Adjustment or the KPI Metrics would have resulted in an increase in the Applicable Percentages for the Revolving Loans incurred by DEI and the Facility Fee for Commitments under the DEI Sublimit for any period, the Borrowers shall be obligated to pay to the Administrative Agent for the account of the applicable Lenders, promptly on demand by the Administrative Agent (or, after the occurrence of an actual or deemed entry of an order for relief with respect to any Borrower under the Bankruptcy Code (or any comparable event under non-U.S. debtor relief laws), automatically and without further action by the Administrative Agent or any Lender), but in any event within 10 Business Days after the Borrowers have received written notice of, or have agreed in writing that there was, a Pricing Certificate Inaccuracy, an amount equal to the excess of (1) the amount of interest and fees that should have been paid for such period over (2) the amount of interest and fees actually paid for such period. If a Borrower becomes aware of any Pricing Certificate Inaccuracy and, in connection therewith, if a proper calculation of the Sustainability Margin Adjustment, Sustainability Fee Adjustment or the KPI Metrics would have resulted in a decrease in the Applicable Percentages for the Revolving Loans incurred by DEI and the Facility Fee for Commitments under the DEI Sublimit for any period, then, upon receipt by the Administrative Agent of notice from the Borrowers of such Pricing Certificate Inaccuracy (which notice shall include corrections to the calculations of the Sustainability Margin Adjustment, Sustainability Fee Adjustment or the KPI Metrics, as applicable), commencing on the Business Day following receipt by the Administrative Agent of such notice, the Applicable Percentages for the Revolving Loans incurred by DEI and the Facility Fee for Commitments under the DEI Sublimit shall be adjusted to reflect the corrected calculations of the Sustainability Margin Adjustment, Sustainability Fee Adjustment or the KPI Metrics, as applicable. (e) It is understood and agreed that any Pricing Certificate Inaccuracy shall not constitute a Default or Event of Default; provided, that, the Borrowers comply with the terms of this Section 1.7 with respect to such Pricing Certificate Inaccuracy. Notwithstanding anything to the contrary herein, unless such amounts shall be due upon the occurrence of an actual or deemed entry of an order for relief with respect to a Borrower under the Bankruptcy Code (or any comparable event under non-U.S. debtor relief laws), (a) any additional amounts required to be paid pursuant the immediate preceding paragraph shall not be due and payable until the date that is 10 Business Days after a written demand is made for such payment by the Administrative Agent in accordance with such paragraph, (b) any nonpayment of such additional amounts prior to or upon such demand for payment by Administrative Agent shall not constitute a Default (whether retroactively or otherwise) and (c) none of such additional amounts shall be deemed overdue prior to the date that is 10 Business Days after such a demand or shall accrue interest at the rate provided in Section 3.1(b) prior to the date that is 10 Business Days after such a demand. (f) Each party hereto hereby agrees that neither the Administrative Agent nor the Co-Sustainability Structuring Agent shall have any responsibility for (or liability in respect of) reviewing, auditing or otherwise evaluating any calculation by any Borrower of any Sustainability Margin Adjustment or Sustainability Fee Adjustment (or any of the data or computations that are part of or related to any such calculation) set forth in any Pricing Certificate (and the Administrative Agent and the Co-Sustainability Structuring Agent may rely conclusively on any such certificate, without further inquiry). (g) As soon as available and in any event within 120 days following the end of each calendar year (commencing with the calendar year ending December 31, 2021), a Pricing Certificate for the most recently-ended calendar year may be provided by DEI as set forth in this Section 1.7; provided, that, for any calendar year the Borrowers may elect not to deliver a Pricing Certificate, such election shall not constitute a Default or Event of Default (but such failure to so deliver a Pricing Certificate by the end of such 120-day period shall result in the Sustainability Margin Adjustment and Sustainability Fee Adjustment being applied as set forth in Section 1.7(c). (h) In the event Borrowers or any of their Subsidiaries acquire or divest a business, facility or Subsidiary with Capacity in excess of 100MW, the Renewable Energy Generation Capacity Percentage Target and the Renewable Energy Generation Capacity Percentage Threshold shall be adjusted to account for such acquisition or divestiture such that the Renewable Energy Generation Capacity Percentage Target and the Renewable Energy Generation Capacity Percentage Threshold remain neutral to such acquisition or disposition in a manner and methodology that are the same as those used in determining the original Renewable Energy Generation Capacity Percentage Target and the Renewable Energy Generation Capacity Percentage Threshold. The Borrowers shall deliver to the Administrative Agent and the Lenders a certificate that (i) calculates in reasonable detail such adjusted Renewable Energy Generation Capacity Percentage Target and Renewable Energy Generation Capacity Percentage Threshold and (ii) restates Exhibit 1.7-1 with such adjusted amounts, and, if Lenders constituting Required Lenders have not objected to such adjusted Renewable Energy Generation Capacity Percentage Target and Renewable Energy Generation Capacity Percentage Threshold within 5 Business Days of such delivery, then Exhibit 1.7-1 shall be deemed amended to reflect such adjusted Renewable Energy Generation Capacity Percentage Target and Renewable Energy Generation Capacity Percentage Threshold.

Draft better contracts in just 5 minutes Get the weekly Law Insider newsletter packed with expert videos, webinars, ebooks, and more!