Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 530 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1154. Min Range: -3.1154

Max Act: 3.1154. Min Act: -0.1700

Data Index: 6083251 (Open Web Text)

Max Activating Token Index: 832

Text #1

Max Range: 3.1154. Min Range: -3.1154

Max Act: 3.0717. Min Act: -0.1700

Data Index: 5583641 (Open Web Text)

Max Activating Token Index: 461

Text #2

Max Range: 3.1154. Min Range: -3.1154

Max Act: 3.0064. Min Act: -0.1700

Data Index: 6254174 (Open Web Text)

Max Activating Token Index: 236

Text #3

Max Range: 3.1154. Min Range: -3.1154

Max Act: 3.0043. Min Act: -0.1700

Data Index: 2987014 (Open Web Text)

Max Activating Token Index: 1013

Text #4

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9928. Min Act: -0.1700

Data Index: 7719743 (Open Web Text)

Max Activating Token Index: 262

Text #5

Max Range: 3.1154. Min Range: -3.1154

Max Act: 3.0006. Min Act: -0.1700

Data Index: 7940929 (Open Web Text)

Max Activating Token Index: 461

Text #6

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9882. Min Act: -0.1700

Data Index: 1290202 (Open Web Text)

Max Activating Token Index: 634

Text #7

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9567. Min Act: -0.1700

Data Index: 8206251 (Open Web Text)

Max Activating Token Index: 821

Text #8

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9493. Min Act: -0.1700

Data Index: 2809554 (Open Web Text)

Max Activating Token Index: 613

Text #9

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9170. Min Act: -0.1700

Data Index: 299855 (Open Web Text)

Max Activating Token Index: 412

Text #10

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9315. Min Act: -0.1700

Data Index: 2934293 (Open Web Text)

Max Activating Token Index: 564

Text #11

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9353. Min Act: -0.1700

Data Index: 1019932 (Open Web Text)

Max Activating Token Index: 410

Text #12

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9297. Min Act: -0.1700

Data Index: 1266078 (Open Web Text)

Max Activating Token Index: 47

Text #13

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9109. Min Act: -0.1700

Data Index: 4603396 (Open Web Text)

Max Activating Token Index: 573

Text #14

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9148. Min Act: -0.1700

Data Index: 7201099 (Open Web Text)

Max Activating Token Index: 544

Text #15

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9109. Min Act: -0.1700

Data Index: 7061177 (Open Web Text)

Max Activating Token Index: 505

Text #16

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9177. Min Act: -0.1700

Data Index: 329032 (Open Web Text)

Max Activating Token Index: 781

Text #17

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.8920. Min Act: -0.1700

Data Index: 7110489 (Open Web Text)

Max Activating Token Index: 293

Text #18

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.8972. Min Act: -0.1700

Data Index: 6441597 (Open Web Text)

Max Activating Token Index: 877

Text #19

Max Range: 3.1154. Min Range: -3.1154

Max Act: 2.9004. Min Act: -0.1700

Data Index: 6971272 (Open Web Text)

Max Activating Token Index: 230