Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2536 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.5064. Min Act: -0.1700

Data Index: 3406086 (Open Web Text)

Max Activating Token Index: 337

Text #1

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.4297. Min Act: -0.1700

Data Index: 1466202 (Open Web Text)

Max Activating Token Index: 133

Text #2

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.4236. Min Act: -0.1700

Data Index: 2681417 (Open Web Text)

Max Activating Token Index: 363

Text #3

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.3644. Min Act: -0.1700

Data Index: 1281863 (Open Web Text)

Max Activating Token Index: 213

Text #4

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.2592. Min Act: -0.1700

Data Index: 3908290 (Open Web Text)

Max Activating Token Index: 100

Text #5

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.2444. Min Act: -0.1700

Data Index: 3072266 (Open Web Text)

Max Activating Token Index: 393

Text #6

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.2825. Min Act: -0.1700

Data Index: 1018306 (Open Web Text)

Max Activating Token Index: 1009

Text #7

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.2715. Min Act: -0.1700

Data Index: 3772565 (Open Web Text)

Max Activating Token Index: 583

Text #8

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.2500. Min Act: -0.1700

Data Index: 1028221 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.2171. Min Act: -0.1700

Data Index: 3148640 (Open Web Text)

Max Activating Token Index: 158

Text #10

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1828. Min Act: -0.1700

Data Index: 5295757 (Open Web Text)

Max Activating Token Index: 731

Text #11

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1460. Min Act: -0.1700

Data Index: 3762403 (Open Web Text)

Max Activating Token Index: 102

Text #12

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1683. Min Act: -0.1700

Data Index: 7222190 (Open Web Text)

Max Activating Token Index: 832

Text #13

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.0884. Min Act: -0.1700

Data Index: 154242 (Open Web Text)

Max Activating Token Index: 618

Text #14

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1169. Min Act: -0.1700

Data Index: 2697292 (Open Web Text)

Max Activating Token Index: 780

Text #15

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1486. Min Act: -0.1700

Data Index: 6266147 (Open Web Text)

Max Activating Token Index: 844

Text #16

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1191. Min Act: -0.1700

Data Index: 6352834 (Open Web Text)

Max Activating Token Index: 274

Text #17

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1339. Min Act: -0.1700

Data Index: 3365903 (Open Web Text)

Max Activating Token Index: 821

Text #18

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.1298. Min Act: -0.1700

Data Index: 1050986 (Open Web Text)

Max Activating Token Index: 279

Text #19

Max Range: 5.5064. Min Range: -5.5064

Max Act: 5.0982. Min Act: -0.1700

Data Index: 4405745 (Open Web Text)

Max Activating Token Index: 352