Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2418 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.7832. Min Act: -0.1700

Data Index: 350652 (Open Web Text)

Max Activating Token Index: 444

Text #1

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.5552. Min Act: -0.1700

Data Index: 98036 (Open Web Text)

Max Activating Token Index: 477

Text #2

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.4899. Min Act: -0.1700

Data Index: 1325262 (Open Web Text)

Max Activating Token Index: 990

Text #3

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.4477. Min Act: -0.1700

Data Index: 2487690 (Open Web Text)

Max Activating Token Index: 969

Text #4

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.4437. Min Act: -0.1700

Data Index: 5480222 (Open Web Text)

Max Activating Token Index: 475

Text #5

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.3745. Min Act: -0.1700

Data Index: 7218903 (Open Web Text)

Max Activating Token Index: 449

Text #6

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.2920. Min Act: -0.1700

Data Index: 8532366 (Open Web Text)

Max Activating Token Index: 988

Text #7

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.3655. Min Act: -0.1700

Data Index: 2699829 (Open Web Text)

Max Activating Token Index: 483

Text #8

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.1922. Min Act: -0.1700

Data Index: 449882 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.1704. Min Act: -0.1700

Data Index: 2526773 (Open Web Text)

Max Activating Token Index: 987

Text #10

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.0900. Min Act: -0.1700

Data Index: 4730302 (Open Web Text)

Max Activating Token Index: 958

Text #11

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.2002. Min Act: -0.1700

Data Index: 2715750 (Open Web Text)

Max Activating Token Index: 989

Text #12

Max Range: 8.7832. Min Range: -8.7832

Max Act: 7.9276. Min Act: -0.1700

Data Index: 6105727 (Open Web Text)

Max Activating Token Index: 1003

Text #13

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.0177. Min Act: -0.1700

Data Index: 2043876 (Open Web Text)

Max Activating Token Index: 431

Text #14

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.0477. Min Act: -0.1700

Data Index: 2924696 (Open Web Text)

Max Activating Token Index: 965

Text #15

Max Range: 8.7832. Min Range: -8.7832

Max Act: 8.0270. Min Act: -0.1700

Data Index: 7577044 (Open Web Text)

Max Activating Token Index: 964

Text #16

Max Range: 8.7832. Min Range: -8.7832

Max Act: 7.9222. Min Act: -0.1700

Data Index: 572237 (Open Web Text)

Max Activating Token Index: 1000

Text #17

Max Range: 8.7832. Min Range: -8.7832

Max Act: 7.9550. Min Act: -0.1700

Data Index: 6035157 (Open Web Text)

Max Activating Token Index: 471

Text #18

Max Range: 8.7832. Min Range: -8.7832

Max Act: 7.8677. Min Act: -0.1700

Data Index: 4406970 (Open Web Text)

Max Activating Token Index: 492

Text #19

Max Range: 8.7832. Min Range: -8.7832

Max Act: 7.8176. Min Act: -0.1700

Data Index: 2753113 (Open Web Text)

Max Activating Token Index: 988