Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2019 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.2339. Min Act: -0.1388

Data Index: 1760849 (Open Web Text)

Max Activating Token Index: 1020

Text #1

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.2205. Min Act: -0.1262

Data Index: 2881365 (Open Web Text)

Max Activating Token Index: 1021

Text #2

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.2163. Min Act: -0.1172

Data Index: 7086972 (Open Web Text)

Max Activating Token Index: 1022

Text #3

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.2041. Min Act: -0.0952

Data Index: 1223871 (Open Web Text)

Max Activating Token Index: 1022

Text #4

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1952. Min Act: -0.1361

Data Index: 5515589 (Open Web Text)

Max Activating Token Index: 1021

Text #5

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.2007. Min Act: -0.1225

Data Index: 7471594 (Open Web Text)

Max Activating Token Index: 1022

Text #6

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1934. Min Act: -0.0823

Data Index: 3879768 (Open Web Text)

Max Activating Token Index: 1022

Text #7

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1838. Min Act: -0.1464

Data Index: 6049765 (Open Web Text)

Max Activating Token Index: 1019

Text #8

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1880. Min Act: -0.1245

Data Index: 1724052 (Open Web Text)

Max Activating Token Index: 1020

Text #9

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1850. Min Act: -0.0180

Data Index: 6070010 (Open Web Text)

Max Activating Token Index: 1021

Text #10

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1836. Min Act: -0.0744

Data Index: 7855559 (Open Web Text)

Max Activating Token Index: 1022

Text #11

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1851. Min Act: -0.1237

Data Index: 6383399 (Open Web Text)

Max Activating Token Index: 1019

Text #12

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1775. Min Act: -0.1100

Data Index: 7220479 (Open Web Text)

Max Activating Token Index: 1020

Text #13

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1762. Min Act: -0.0583

Data Index: 8322084 (Open Web Text)

Max Activating Token Index: 1021

Text #14

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1628. Min Act: -0.1000

Data Index: 7733145 (Open Web Text)

Max Activating Token Index: 1021

Text #15

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1614. Min Act: -0.1066

Data Index: 674519 (Open Web Text)

Max Activating Token Index: 1022

Text #16

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1786. Min Act: -0.1226

Data Index: 6500382 (Open Web Text)

Max Activating Token Index: 1022

Text #17

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1782. Min Act: -0.1341

Data Index: 5420669 (Open Web Text)

Max Activating Token Index: 1015

Text #18

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1682. Min Act: -0.0855

Data Index: 2218685 (Open Web Text)

Max Activating Token Index: 1019

Text #19

Max Range: 1.2339. Min Range: -1.2339

Max Act: 1.1747. Min Act: -0.1211

Data Index: 198721 (Open Web Text)

Max Activating Token Index: 1022