Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 514 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.5225. Min Act: -0.1700

Data Index: 3959686 (Open Web Text)

Max Activating Token Index: 890

Text #1

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.3826. Min Act: -0.1700

Data Index: 7651584 (Open Web Text)

Max Activating Token Index: 407

Text #2

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.2859. Min Act: -0.1700

Data Index: 1660807 (Open Web Text)

Max Activating Token Index: 397

Text #3

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.2466. Min Act: -0.1700

Data Index: 6441990 (Open Web Text)

Max Activating Token Index: 433

Text #4

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.2113. Min Act: -0.1700

Data Index: 1927630 (Open Web Text)

Max Activating Token Index: 741

Text #5

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.2170. Min Act: -0.1700

Data Index: 835996 (Open Web Text)

Max Activating Token Index: 421

Text #6

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.2275. Min Act: -0.1700

Data Index: 1583212 (Open Web Text)

Max Activating Token Index: 877

Text #7

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1946. Min Act: -0.1700

Data Index: 3418965 (Open Web Text)

Max Activating Token Index: 443

Text #8

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1669. Min Act: -0.1700

Data Index: 1974801 (Open Web Text)

Max Activating Token Index: 321

Text #9

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1799. Min Act: -0.1700

Data Index: 2489978 (Open Web Text)

Max Activating Token Index: 961

Text #10

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1739. Min Act: -0.1700

Data Index: 8744270 (Open Web Text)

Max Activating Token Index: 37

Text #11

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1750. Min Act: -0.1700

Data Index: 6096752 (Open Web Text)

Max Activating Token Index: 50

Text #12

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1789. Min Act: -0.1700

Data Index: 6881996 (Open Web Text)

Max Activating Token Index: 935

Text #13

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1301. Min Act: -0.1700

Data Index: 813793 (Open Web Text)

Max Activating Token Index: 905

Text #14

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1494. Min Act: -0.1700

Data Index: 180369 (Open Web Text)

Max Activating Token Index: 560

Text #15

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1317. Min Act: -0.1700

Data Index: 5019494 (Open Web Text)

Max Activating Token Index: 98

Text #16

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1485. Min Act: -0.1700

Data Index: 4631161 (Open Web Text)

Max Activating Token Index: 516

Text #17

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1170. Min Act: -0.1700

Data Index: 4148517 (Open Web Text)

Max Activating Token Index: 969

Text #18

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1076. Min Act: -0.1700

Data Index: 6048622 (Open Web Text)

Max Activating Token Index: 244

Text #19

Max Range: 3.5225. Min Range: -3.5225

Max Act: 3.1075. Min Act: -0.1700

Data Index: 5019246 (Open Web Text)

Max Activating Token Index: 664