Evaluation of experiments with NER-based re-ranking Sample Clauses

Evaluation of experiments with NER-based re-ranking. We have tested different re-ranking strategies in the two evaluation datasets: the Xxxx- Xxxx news corpus and the #Microposts2014 tweets corpus. Table 1 shows some of the results we obtained. The first row shows the precision and recall attained by DBSpotlight, with no confidence threshold specified. In this baseline case, precision is lowest and recall is highest. Combination Precision in Detection Recall in Detection F-measure in Detection Accuracy in Classification of Detected Items DBSpotlight 0.2546 0.7477 0.3799 0.7272 DBSpotlight (c > 0.9) 0.3002 0.6871 0.4178 0.7639 Re-ranked DBSpotlight 0.2546 0.7477 0.3799 0.6979 Re-ranked DBSpotlight (c > 0.9) 0.3140 0.7342 0.4399 0.7369 Table 1: Results of re-ranking on the Xxxx-Xxxx (news) dataset The other rows combine a confidence threshold of 0.93 with the re-ranking heuristics explained above. The best F-measure (0.4399) in detection is attained with c>0.9 and re-ranking. However the most remarkable result here appears in the third row: re- ranking is actually detrimental to accuracy in classification, while not improving precision and recall in detection. The following example illustrates what is actually happening: Following are highlights of the midday briefing by the European Commission on Wednesday.  Stanford NER correctly annotates European Commission as ORGANISATION.  DBPedia Spotlight provides this ranked list of candidate annotations, with their probabilities. European_Comission (0.9999999995178541) ECHO_(European_Commission) (4.532487558872544E-10) Barroso_Commission (6.689508820986744E-21) Directorate- General_for_Education_and_Culture_(European_Commission) (1.9533030489271525E-22) European_Atomic_Energy_Community (2.8913626187425216E-11) Internationalization_of_the_Danube_River (5.449914081431767E-27) European_Commissioner_for_Energy (4.627817959997166E-26) EuropeAid_Development_and_Cooperation (5.7615592731035274E-24) The good one is, correctly, the first one, but, unfortunately, its dbtype is empty, so the re-ranking strategy incorrectly picks the first one with a dbtype compatible with ORGANISATION (the one in bold).
Sample 1
AutoNDA by SimpleDocs

Related to Evaluation of experiments with NER-based re-ranking

  • Completion of Evaluation Cycle 1. The summative evaluation rating shall be based upon a preponderance of the evidence, assessed in a holistic manner, that is aligned to the Ohio Educator Standards. Only evidence gathered during the walkthroughs and formal observations that are conducted for the current school year may be used.

  • Finalization of Evaluation A Written Report 1 Before the evaluation cycle is final, and not later than May 10, a copy of the formal written evaluation report shall be given to the teacher and a conference shall be held between the teacher and the evaluator.

  • Independent Evaluation Buyer is an experienced and knowledgeable investor in the oil and gas business. Buyer has been advised by and has relied solely on its own expertise and legal, tax, title, reservoir engineering, environmental and other professional counsel concerning this transaction, the Properties, the value thereof and title thereto.

  • Evaluation Cycle Goal Setting and Development of the Educator Plan

  • Evaluation of Students Acknowledging the District’s adopted grading system, the teacher shall maintain the right and responsibility to determine grades and other evaluation of a student. No grade or evaluation shall be changed except by the teacher with the approval of the building administrator.

  • Services and Information for Persons with Limited English Proficiency A. Grantee shall take reasonable steps to provide services and information both orally and in writing, in appropriate languages other than English, to ensure that persons with limited English proficiency are effectively informed and can have meaningful access to programs, benefits and activities. Meaningful access may entail providing language assistance services, including oral interpretation and written translation, if necessary. More information can be found at xxxxx://xxx.xxx.xxx/.

  • Evaluation of Contractor Performance of the Contractor under this Agreement will be evaluated. The evaluation shall be prepared on Contract/Contractor Evaluation Sheet (STD 4), and maintained in the Agreement file. For consultant agreements, a copy of the evaluation will be sent to the Department of General Services, Office of Legal Services, if it is negative and over $5,000.

  • Form B - Contractor’s Annual Employment Report Throughout the term of the Contract by May 15th of each year the Contractor agrees to report the following information to the State Agency awarding the Contract, or if the Contractor has provided Contract Employees pursuant to an OGS centralized Contract, such report must be made to the State Agency purchasing from such Contract. For each covered consultant Contract in effect at any time between the preceding April 1st through March 31st fiscal year or for the period of time such Contract was in effect during such prior State fiscal year Contractor reports the:

  • Project Monitoring Reporting Evaluation A. The Project Implementing Entity shall monitor and evaluate the progress of its activities under the Project and prepare Project Reports in accordance with the provisions of Section 5.08(b) of the General Conditions and on the basis of indicators agreed with the Bank. Each such report shall cover the period of one

  • HHS Single Audit Unit will notify Grantee to complete the Single Audit Determination Form If Grantee fails to complete the form within thirty (30) calendar days after receipt of notice, Grantee maybe subject to sanctions and remedies for non-compliance.

Time is Money Join Law Insider Premium to draft better contracts faster.